stop referer spam

How to block referrer spam




A guide to block referrer spam

Detailed  instructions on how to effectively start blocking referrer spam from your website and to stop affecting analytics reports.

Running a blog can be a  rewarding and enriching experience which has the possibility to create opportunities. However, as with all things in life there are a lot of idiots out there who try to do their utmost to destroy, create obstacles and generally want to be nothing more than a menace to society in general,  Referrer spammers fall into this category. 

Referer spammers are the graffiti artists of the internet, seeking to leave their illiterate markings everywhere.

Learn more about Referer Spam

How to tell if your site is targeted by referral spam

If you examine your Google Analytics report and see an increase in referrers coming from any of the sample sites:

  • semalt.com
  • priceg.com
  • hulfingtonpost.com
  • bestwebsitesawards.com
  • cenoval.ru

There is a high chance likelihood your site has been targetted you and you need to take evasive action.

Using WordPress ?

I have developed a WP Plugin to stop referer spam visits to your WordPress website

Download free wordpress plugin – Stop Web Crawlers

Fortunately there are a few solutions you can implement to combat the effects of referer spam.

Using your .htaccess file to combat referrer spam

If your website is hosted using an apache webserver you’ll need to update your .htaccess file.

A useful resource learn how to update your .htaccess file and to ensure you update it with a list of most known spammers and bad bots is to read Fight Blog Spam with Apache.

A new tactic of using SetEnvIfNoCase instead of RewriteCond seems to be quite effective against referral spammers.

Learn about types of referrer spam




Google Analytics Filters

In many cases the spammers bots never actually visit your site other. On some occasions these bots may initially only visit your site to retrieve your google analytics code or account number, once they have this they will spoof visits to your site to pollute your analytics report with false visits.

Fortunately Google Analytics provides you with utilities to eliminate spammers from your analytics report. The only downside is that they will not be applied retrospectively!  The first step is to enable automatic known bot filtering.

To do this you will need to log into your Google Analytics Account and go to the Admin area then navigate to View Settings.

botfilter2

Then select the “Exclude all hits from known bots and spiders”

BotFilterUnfortunately this will step will not filter out all the spammers and we still need to do some manual configuration

Manually Configure Bot Filtering

We do this by accessing the Filters view

botfilter3 Click on add New Filter

We now Create new Filter and Custom and select Custom and Exclude and Referral from the Filter Field. 

botfilter4

You can manually enter each URL you want to filter out one by one, or you can do what I do and put them all into one string using a POSIX regular expression
Personally I have created to filter exlusions list on my analytics

Spammer filter list
Spammer filter list extension
 

You can add more to this string as you go remembering to just use the | (Pipe) at end of the string before the closing bracket  and adding the Url , and escaping the . with a \ (blackslash)

** A restriction by google that the length of your Regular expression can only be as long as 255 characters so you may need to create another filter and add new URLS as you find them **
How to filter referrer spam from google analytics historical data

Blocking Referer spam IP Address

An additional precautionary step I always take is to block any traffic coming from the IP address of the known referer spammers is denied access to my website. It is very simple to do this your .htaccess file

The first line “Order allow, deny” tells the web server the “Order”  in which the Allow and Deny directive will be evaluated. The rest of the command simply implies: Allow access to all hosts that are not present in the Deny from list and are present in the Allow from list. The allow, deny order Allow list is looked up first and then the web server checks the deny from list.

Google Analytics IP Exclusion List

The final step I also go through to ensure that I can ensure that I experience no more problems with referer spam is that I exclude all known IP addresses from any future reports.  To do this I follow the same approach as in Google Analytics Filters, however instead of custom I choose Predefined.

IP Filter

I will then create a filter for each  IP Address  associated with any Referer spammer website i.e. 78.110.60.230,  64.13.232.232

If you have any further questions please ask in the comments section below or contact me and I will endeavour to respond as soon as I can.

Gary Woodfine

Freelance Full Stack Developer at threenine.co.uk
Helps businesses by improving their technical proficiencies and eliminating waste from the software development pipelines.

A unique background as business owner, marketing, software development and business development ensures that he can offer the optimum business consultancy services across a wide spectrum of business challenges.
π