A guide to block referrer spam
Detailed instructions on how to effectively start blocking referrer spam from your website and to stop affecting analytics reports.
Running a blog can be a rewarding and enriching experience which has the possibility to create opportunities. However, as with all things in life there are a lot of idiots out there who try to do their utmost to destroy, create obstacles and generally want to be nothing more than a menace to society in general, Referrer spammers fall into this category.
Referer spammers are the graffiti artists of the internet, seeking to leave their illiterate markings everywhere.
How to tell if your site is targeted by referral spam
If you examine your Google Analytics report and see an increase in referrers coming from any of the sample sites:
- semalt.com
- priceg.com
- hulfingtonpost.com
- bestwebsitesawards.com
- cenoval.ru
There is a high chance likelihood your site has been targetted you and you need to take evasive action.
Advice
Using WordPress ?
I have developed a WP Plugin to stop referer spam visits to your WordPress website
Download free wordpress plugin - Stop Web Crawlers
Fortunately there are a few solutions you can implement to combat the effects of referer spam.
- Using your .htaccess file to combat referrer spam
- Google Analytics Filters
- Blocking IP Address
- Google Analytics IP Exclusion List
Using your .htaccess file to combat referrer spam
If your website is hosted using an apache webserver you'll need to update your .htaccess file.
A useful resource learn how to update your .htaccess file and to ensure you update it with a list of most known spammers and bad bots is to read Fight Blog Spam with Apache.
A new tactic of using SetEnvIfNoCase instead of RewriteCond seems to be quite effective against referral spammers.
Learn about types of referrer spam
Google Analytics Filters
In many cases the spammers bots never actually visit your site other. On some occasions these bots may initially only visit your site to retrieve your google analytics code or account number, once they have this they will spoof visits to your site to pollute your analytics report with false visits.
Fortunately Google Analytics provides you with utilities to eliminate spammers from your analytics report. The only downside is that they will not be applied retrospectively! The first step is to enable automatic known bot filtering.
To do this you will need to log into your Google Analytics Account and go to the Admin area then navigate to View Settings.
Then select the "Exclude all hits from known bots and spiders"
Unfortunately this will step will not filter out all the spammers and we still need to do some manual configuration
Manually Configure Bot Filtering
We do this by accessing the Filters view
Click on add New Filter
We now Create new Filter and Custom and select Custom and Exclude and Referral from the Filter Field.
You can manually enter each URL you want to filter out one by one, or you can do what I do and put them all into one string using a POSIX regular expression
Personally I have created to filter exlusions list on my analytics
Spammer filter list
Spammer filter list extension
You can add more to this string as you go remembering to just use the | (Pipe) at end of the string before the closing bracket and adding the Url , and escaping the . with a \ (blackslash)
Blocking Referer spam IP Address
An additional precautionary step I always take is to block any traffic coming from the IP address of the known referer spammers is denied access to my website. It is very simple to do this your .htaccess file
The first line “Order allow, deny†tells the web server the “Order†in which the Allow and Deny directive will be evaluated. The rest of the command simply implies: Allow access to all hosts that are not present in the Deny from list and are present in the Allow from list. The allow, deny order Allow list is looked up first and then the web server checks the deny from list.
Google Analytics IP Exclusion List
The final step I also go through to ensure that I can ensure that I experience no more problems with referer spam is that I exclude all known IP addresses from any future reports. To do this I follow the same approach as in Google Analytics Filters, however instead of custom I choose Predefined.
I will then create a filter for each IP Address associated with any Referer spammer website i.e. 78.110.60.230, 64.13.232.232
- What is this Directory.Packages.props file all about? - January 25, 2024
- How to add Tailwind CSS to Blazor website - November 20, 2023
- How to deploy a Blazor site to Netlify - November 17, 2023