Keith & Annabel Morrigan on Friday, 20 January, 2012 at 18:19:25 (modified: 23 January, 2012 at 13:00:00)
Our first blog entry becomes Worldwide news!
We are stunned by the response to our first blog post Rumor turns Rogue - McAfee Software Hacked! on our McAfee problems and guidance for other affected users. Power to the internet!!
Thank you to all those reporters, with special thanks to Elinor Mills for her CNET article and to Leo Kelion for his BBC article.
Other than one small blog post, the global threat intelligence systems run by McAfee report all the other security threats, but not their own! "Sorry" seems to be the hardest word!
UPDATE 23/01/12: Although no major announcement from McAfee, they have now published Security Bulletin SB10018 on this issue.
The McAfee problem in summary
In our post Rumor turns Rogue - McAfee Software Hacked! we explained that the McAfee SaaS Total Protection service, intended to protect computers from internet hazards, was itself allowing others to send spam (amongst other things) from our server by acting as an "open proxy".
Many others were also believed to be affected, with more seen every day, so we advised that the Rumor Service (part of the McAfee software) should be disabled, and firewall rules changed to block incoming tcp traffic on port 6515, until a McAfee patch was issued.
Affected users could have internet connection problems, problems with their ISP due to the huge traffic caused and long term issues with blacklisting and returned emails.
McAfee have issued a patch
A McAfee patch has been issued through the automatic update procedure. According to David Marcus, Director of Security Research for McAfee Labs on his blog, "We have mitigating factors already in place that reduce risk" and "all affected customers will automatically receive the patch when it is released" which "will close this relay capability" (as well as fix an ActiveX issue).
Sure enough, this morning we found that one Vista PC (not suffering from the open proxy issue) had been automatically updated. Unfortunately our server, which had been affected, had not been updated.
From the number of port 6515 open proxies still being listed/found on Hinky Dink's Proxy Obsession as of today, clearly there are still other affected users without the patch yet.
UPDATE 23/01/12: Many new 6515 open proxies are still being found!!
Our experience with the patch upgrade
We disabled our Rumor service and installed 6515 port blocking firewall rules on 5th January, but highlighted that the Rumor service could be restarted as part of an automatic upgrade.
With these rules in place, and with monitoring of the Rumor service to switch it off again if it restarted, automatic updates continued as normal until 13th January, since when no successful updates have occurred.
We are assuming that prevention of upgrades, with the corresponding risk of restarting the Rumor service, was one of the "mitigating factors" mentioned on the McAfee blog.
We had to restart the updating service and carry out a manual upgrade to install the patch, so have included instructions below for any others with the same problem.
Ensuring you have the latest patch
Either double click on the McAfee shield symbol in the System Tray (normally bottom right corner of the screen), or right click on it and select Open Console from the context menu.
On the Action Menu dropdown list, select Product Details.
In the Security Center Communication section it should say Product version : 5.2.3 Patch 4
If it shows Product version : 5.2.3 Patch 3 you are not patched yet.
To start a manual update, right click on the McAfee shield symbol in the System Tray and select Update Now from the context menu. This should open a dialog box showing the update progress.
For us this did not work. Initially no dialog box opened - on a later attempt, the dialog box did open but froze with a message about checking our internet connection.
Our first thought was that our recommended firewall rules were blocking the update, which caused some concern as we did not want to be responsible for providing duff advice!
Through trial and error we determined that the firewall rules were not blocking the update, but that the update process just wasn't working (perhaps the "mitigating factors" had tied it up in knots).
So, if your update doesn't work try this:
Close the McAfee console window, if it is still open.
Start > Administrative Tools > Services
Press Continue to “Windows needs your permission to continue” (if it comes up)
Click on the service “McAfee Virus and Spyware Protection Service” described as “Controls scanning and updating activities at the desktop for Virus and Spyware Protection Services.”.
Look at the top of the empty border to the left of the list of services where it should show the McAfee service name with Stop/Restart the service links and press the Restart link.
You should see a Service Control dialog box showing progress of the service being stopped/started before disappearing.
Even after restarting the service, our system failed to carry out an automatic update and a manual update was required.
Start a manual update by right clicking on the McAfee shield symbol in the System Tray and select Update Now from the context menu. This should open a dialog box showing the update progress.
Once the update is complete, check again that you have the right version. We have found the automatic update to be working since our manual update.
Does the patch work
In a word, YES!
Although reluctant to cause further spamming, we did a test for a short period by restoring the original firewall rules with the Rumor service enabled and carefully monitored our router.
Even two weeks after blocking incoming 6515 connections, we are still getting up to 10 connection requests a second on port 6515 (it was more like a hundred at peak), so it was easy to see if any would be passed on.
We were pleased to see despite hundreds on incoming connections we had no outgoing connections (except for netbios-ns calls where windows is trying to identify the callers).
Because we were affected, still have numerous incoming 6515 calls and need to minimise the traffic these can cause, we will continue to keep the firewall rules in place, and will disable the Rumor Service whenever it pops to life again.
Our ISP indicated that our traffic had returned largely to normal when Rumor was off and not answering incoming calls - we are not sure what counts as traffic! If Rumor is on and listening but 6515 is blocked on our firewall, then the firewall logs quickly fill with DROP messages - does this count as traffic? Again, if Rumor is on and listening with no firewall blocks, does this count as traffic? Or the resulting netbios-ns calls (which we will block)?
Now to try and get back to normal!
Now we know we are secure again, we can try to get back to normal, gradually restoring various functions that we had to suspend, such as our Google Shopping listings and our advertising.
We have been tied up with this, in one way or another, since the beginning of the year. Hopefully we can even get back to our mosaics!
Some aspects will be difficult for us to fix, such as the rating downgrading/blacklisting of emails and reputation damage on security listings all over the world. For some time we are likely to never know who we can email without it being returned, or marked as Junk Mail.
We are still hoping that McAfee will help to restore our reputation, which they have damaged - we will have to wait and see!