I’ve also found Lloyd Borrett posting the same info here.

If I hear any further confirmation from AVG, I’ll let you know, until then we’ll see what happens on the 9th.  Interestingly Lloyd doesn’t see to know what day it is, either that or my calendar is wrong -

In working with the web master community, AVG has responded immediately and on Tuesday, July 9th, AVG will issue a product modification to address the spikes that a few individuals have seen with their web traffic.

Tuesday? I make it Wednesday, ah well, we might see something Tuesday or Wednesday then

Nice use of “a few individuals” there Lloyd, I wouldn’t class it as a few myself.

Now confirmed by Pat Bitton at AVG, see comment below.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

This has been confirmed by Roger Thompson in the AVG Webscanning Google Group, to which I’m a member.  When asked if it was true Roger said -

..Yes, it’s true, but it’s not quite what it seems. It was a planned
service release that was already in the works when we found out about
the issue, and fixed some other critical issues at the same time.

It changes the User Agent string to SV1, but it leaves some of the
other request headers so that they’re different enough that you can
still parse them out of stats if you want to..

I personally don’t really see this as a “new” user agent, as I’m already filtering my logs for this one.  From what I’ve read, this is the original agent used by LinkScanner before they were purchased by AVG, so it’s already mentioned in my initial AVG log spam post.

That aside, it’s still not a great move from AVG at this point.  They are supposed to be working with the community (myself included) to resolve this from a webmasters point of view, switching (well mixing in another) user agent at this point is a little silly.

At this point I need to credit Michael Ducy who tipped me off this change yesterday, he however claims it’s affecting the free version also -

AVG changed the user agent with the latest release. They now use “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”. I confirmed this by downloading and installing AVG free this morning and using wireshark to sniff the traffic.

The saga continues..

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

..I would like to invite you to participate in a group dedicated to discussion
about AVG LinkScanner technology, its advantages as well as potential impact
on web sites, the nature of recent web threats and ways of protecting both
users and web masters. Your input and feedback will be highly appreciated.
The group is closed and by invitation only..

I have accepted the invite but unfortunately now have to be re-approved as I used a different email address (my Google account).

Hopefully we’re getting somewhere on this issue.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

The article fails to mention if this is true for all PPC providers though.  I know clients who’ve seen an increase in Yahoo!/Overture spend since this came about.

Anyone have any further clarification?

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

Hi Adam,

Thanks for your thoughts and offer. I’ve passed this along to our product managers. They are really the right guys to coordinate this.

Sorry for the delay in replying … we’re a bit busy.

Cheers

Roger

Let’s hope this means things are moving from the AVG end and we might get somewhere with sorting this mess.  I’m not holding my breath though.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

…we’ve been working since then to
figure out exactly was going on and how we might fix it with the best
results for everyone. Your offer to assist is much appreciated, and I’m
copying this reply to Roger Thompson* so he and his team can contact you
directly…

So, it looks like we’re getting somewhere and hopefully this’ll mean a change to the way LinkScanner operates to keep webmasters and site owners happy.

* Chief of Research at AVG and designer of LinkScanner while serving as CTO of Exploit Prevention Labs

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

A week has passed since I email The Register about the AVG logfile spam problems, but, after lots of emails between myself and their Internet Editor, they’re now running the story.

I’m a little annoyed that they don’t acknowledge that I brought this to their attention, but ah well. All I really wanted to do was let the community know that this happening and see if together we can sort this.

I’m really pleased to see that an employee from AVG has stepped up within the comments of the article on El Reg and wants to work with us (webmasters/site owners) to see if we can improve the situation. I’ve already emailed them and await a response.

I’ll keep you posted.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

Public Sub Send(ByVal strTo As String, ByVal strFrom As String, _
   ByVal strSender As String, ByVal strCC As String, _
   ByVal strBCC As String, ByVal strSubject As String, _
   ByVal strBody As String)

Dim MailObj As New System.Net.Mail.MailMessage(strFrom, _
   strTo, strSubject, strBody)

'SPF Stuff
If Not String.IsNullOrEmpty(strSender) And strSender <> strFrom Then
   MailObj.Headers.Add("return-path", strSender)
   MailObj.Headers.Add("reply-to", strFrom)
   MailObj.Sender = New Net.Mail.MailAddress(strSender)
End If

If Not strCC = String.Empty Then
   For Each cc As String In strCC.Split(";")
      MailObj.CC.Add(cc)
   Next
End If

If Not strBCC = String.Empty Then
   For Each bcc As String In strBCC.Split(";")
      MailObj.Bcc.Add(bcc)
   Next
End If

Dim MailClient As New System.Net.Mail.SmtpClient
MailClient.Host = "mailserver"
MailClient.Send(MailObj)

End Sub

Again, as before, this is generally useful for web generated emails (like send a friend forms etc.), simply specify the users address as strFrom and a generic local address (noreply@mydomain.com) as strSender.

I hope it proves useful.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

Note: Awstats is a cross platform web analysis tool, but unfortunately logparser isn’t, this therefore is windows only.

To make life easier, I dropped the logparser files (exe and dll, although I’m not sure you need the dll) directly in to the cgi-bin where Awstats lives on the server. I understand doing this may have security implications, so do this at your own risk.

Open up the config file for your Awstats report (awstats.<config>.conf) and find the LogFile directive

LogFile=”E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log”

It’ll be something like the above, assuming you use daily logs on IIS. We need to change it to

LogFile=”logparser -i:iisw3c -o:w3c -rtp:-1 -stats:off file:rem-avg-spam.sql?logfile=E:/logs/W3SVC2074709632/ex%YY-1%MM-1%DD-1.log |”

This tells Awstats to execute logparser setting any necessary options and passing in the path to the log as before, it then grabs the output from the pipe and processes it.

That’s it!

The contents of my rem-avg-spam.sql file is just

select *
from %logfile%
where not (cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;1813)’
or cs(User-Agent)=’Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1;+SV1)’
and cs(Cookie) is null
and cs(Referer) is null)

I’m now using this for some fairly large logs (100mb+) and it works fine.

I hope this helps.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::

For more information on the user agents used and some background on other similar AV tools, see this LinkScanner, AVG, TrendMicro, 1813 and SV1 post at WebmasterWorld.

:: :: :: :: :: :: :: :: :: :: :: :: :: :: :: ::