Filters in Google Analytics
Google Analytics is a fairly complete webanalytics program, but it has some flaws. As I showed in my article on Google Analytics tweaks, you can overcome many of these flaws with the use of tweaks. In this article I will show what you can do with custom filters and extra profiles on the same website.
What are Analytics filters?
Google Analytics filters allow you to select specific visits or visitors on your website and either show only them, exclude them from your reports, or change the way they show up in your reports.
You can enter filters in the “Analytics Settings > Filter Manager” section of your account when you have administrator rights. Here you can enter filters that “Exclude all traffic from a domain”, “Exclude all traffic from an IP address”, “Include only traffic to a subdirectory” or make your own filter under “Custom filter”.
What can I do with Analytics filters?
The first two filters mentioned before help you to exclude visitors that cloud your reports with unrelevant visitors, for instance visits from your companies own IP address. The subdirectory filter allows you to separate subsections of your website into separate reports. More on separate website profiles later, but first let me explain how you can use the “Custom filter”.
Custom filter
In the custom filter section you can include, exclude or edit information based on many variables before they are simplified and combined in your analytics reports. The most important pieces of information to use in your filters are: Request URI, Visitor IP address and Referrer, but others like country and language can always come in handy. A full list can be found here.
Custom filters can use POSIX regular expressions. This means you define what format or sequence the filter should look for and you can extract information from it. I like using the “Advanced filter” from the “Custom filter” section. There you can use one or two fields and combine extracts from them into another field.
Full referrer URLs
Under “Marketing Optimization > Visitor Segment Performance > Referring Source” or “Marketing Optimization > Marketing Campaign Results > Referral Conversion” Analytics shows only partially where your visitors have come from. For instance the get variables (for instance ?name=analytics-filters) are excluded and it isn’t easy to just enter the referrer URL and visit it. To get the entire referral URL into your report you can use a custom-advanced filter. Here’s how:
- Enter the Filter manager
- Click “+ Add Filter”
- Enter the filter name (like “Full Referral”) and select “Custom filter” from the drop-down box
- From the radio buttons you select “Advanced”
- For “Field A -> Extract A” you select “Referral” from the drop-down box and in the text field you enter “(.*)” (to select the entire URL)
- Leave Field B empty and set “Output To -> Constructor” to “User Defined”. In the text field you enter “$A1″ to refer to the first selection of field A.
It should end up looking something like this:
Within a day full referral information will end up under “Marketing Optimization > Visitor Segment Performance > User-defined”
What have Adwords searchers really typed?
Another flaw in Google Analytics is the report on Google Adwords. Analytics only shows what matching option was triggered by the users search query, not what was exactly typed. In my earlier article on Analytics I showed how to fix this with javascript, but you can also use filters to extract this data. Here’s how that goes:
Copy the following screen to get referral data of cpc clicks into “Custom field 1″ (this field cannot be seen in your reports)
Copy the following screen to combine the data of “Custom field 1″ and the original data of “Campaign term” into one
From now the all reports that include Adwords search terms will also include the used query.
Using extra profiles
You can create extra profiles of the same website and filter specific traffic to only appear in all reports of that profile. I use this option to see if visitors that order something show other behaviour then other visitors, but you can also use it to differentiate Google traffic from Yahoo traffic, or new from returning visitors. This works in all your reports and you don’t need to use any cross segmentation within your reports.
Google Analytics can do almost anything expansive web analytics software can do. You just need to know how to retrieve the data.
March 29th, 2007 at 4:53 pm
Hi Peter,
Thanks for your posts on using Google Analytic filters to find out what the Adwords searchers really typed. I was really looking for a hack like this.
I’ve copied your screenshot, but… it doesn’t seem to work like it should.
I hope you can help:
When looking at my cpc-keyword report, the keywords are listed twice:
keyword 12.000 visits 50,2 pages/visit conversion 2,1%
Keyword, (keyword) 5.000 visits 1,3 pages/visit conversion 0%
When I look at the same report in my original profile the keyword is listed like this:
keyword 17.000 visits 28,8 pages/visit conversion 1,7%
Have you got any idea what goes wrong? For every keyword the listing with brackets show an average pages/visit between 1 and 2 and not even 1 conversion.
Have you got any idea what causes this?
Thanks in advance,
Arnoud
March 30th, 2007 at 1:02 pm
This is one of the problems with editing information Google Analytics uses throughout its reports. Analytics doesn’t take altered data into account and therefore tweaking data should be done with the idea that the data might not be so trustworthy as the original one.
This is why I adviced to use a second profile or place the data in an unused placeholder (like “user defined” or fake page requests in a specific directory)
I tested the reports in some of my accounts and these seem to be OK, but that is no guarantee. At the moment I have no idea what caused it.
April 4th, 2007 at 10:44 pm
Great post!
I never realized how powerful google analytics could be until I read your blog. To make a separate profile to look at just new visitors would you do something like this:
Filter Type: Custom
Exclude
Filter Field: Visitor Type
Filter Patter: (Returning Visitors)
Is that about right or am I missing something?
April 5th, 2007 at 9:10 am
Hello Justin,
When you don’t know what exactly a field returns, you can look at http://www.google.com/support/analytics/bin/answer.py?answer=27218 or test it by forwarding the raw information to the user defined field with a custom filter.
In the new or existing visitors case it is “New Visitor” or “Returning Visitor” without the quotes.
April 5th, 2007 at 7:20 pm
Hi Peter,
At the end of your post you list a possibility to view only adwords traffics in analytics. Can you describe what filter I need to set up to do just that?
April 6th, 2007 at 10:11 am
That should look something like this:
I haven’t tested it because it is only applied to new visitors.
April 17th, 2007 at 4:09 pm
Peter,
I’m attempting to track traffic from a partner company that has ~130 sites. I spoke with the GA guys at SES and they recommended that I set up a seperate profile and use the Custom Filter > Include > Filter Field: Referral and then type the list of the sites I wish to track seperated by a |.
For example: www.siteA.com|www.siteB.com|www.siteC.com so on.
However, the traffic is not reporting accurately.
Because I wish for the text links to count for SEO purposes, I don’t want to add tracking variables into the href. Do you know how to isolate this traffic. Thanks in advance.
April 17th, 2007 at 4:44 pm
For heavier user statistics I’d recommend using Clicktracks or Webtrends (or whatever other software gives good segmentation possibilities). Google Analytics just isn’t build for this stuff.
I’m not sure if I get your question correctly. Do you want to “track 130 sites” or “separate 130 referers” in one account?
What you seem to be doing here is creating a separate account that just tracks traffic from any of the given referers. Excluding traffic from all other referers. But all this traffic is thrown onto one giant heap. For what purpose would you do such a thing?
If that is what you’re trying to do and you get only half the traffic you should get it is always good to see what referer string is returned by analytics. Throw the entire string into the user defined variable, by using a the “Full referrer URLs” custom filter I’ve demonstrated in this article. Then see if your regular expression catches all the referers it should catch or if there are exceptions that you need to tweak the expression for. (Perl regular expressions)
April 17th, 2007 at 5:17 pm
Correct Peter,
I’m attempting to track traffic from any of the given referers. I’ve agreed to pay a monthly fee for this company to place static text links on ~130 sites they control. My goal is to measure if my spend is worth traffic. Ideally, I would like to track the traffic identically as if I had a source variable in the link href on the partner sites.
Again, thank you.
April 17th, 2007 at 9:43 pm
This is the best solution in your case:
With PHP you define all domains in an array and check for them with php. Then you return campaign data to Analytics which can be used throughout many reports. You don’t even need to create a separate profile.
Here it is in a separate text file: checkdomains.txt
May 7th, 2007 at 10:53 pm
I created a separated profile for the filter discussed here.
For the same period of time, I compared data obtained through each profile, in “Overall Keyword Conversion” statistics.
The number of clicks is not matching the data.
Let’s say you have only one keyword, in broad, like “kewyword 1″. The filtered list should contain all the combinations through which visitors comes to the web site - there are 40 of them. But, the unfiltered list contain 120 clicks for the broad match.
Where are the other 80 clicks? :(
May 14th, 2007 at 10:19 am
Multiple people encountered the same problem, but my profile worked just fine. I just tested on someones profile that didn’t work and outputted the raw URL data to the User Defined field. It seems the filter needs some tweaking for exceptions which I will probably do next week.
May 21st, 2007 at 3:32 pm
Tremendous blog and blog post re: analytics filters. Thank you!
A question: If I am using a tracking URL of the format:
http://www.mysite.com?channel=xyz
Will the referring traffic filter you describe above track these specifically? Will it do so regardless of whether the link is placed on site xyz or typed directly into the browser?
Thanks again!!
–Craig
May 21st, 2007 at 5:13 pm
The mentioned filter tracks referers. Something your browser transmits at a page request as extra information, it is not the requested URL itself.
I’m guessing http://www.mysite.com?channelxyz stands for the URL within your website at the recieving end of a campaign. In many cases you want to have both linkpoints and campaign tracking, so there is something extra you need to do.
I won’t go in to everything with great detail, but the way I do it is the following:
- Linkpoints need to be focussed at one unique url. Separate tracking URLs with the same content will be filtered by search engines, so we need to 301 redirect to the one URL that can score. For instance the same url without campaign tracking information.
- Before returning a 301 header to a browser or searchbot, you cannot return any html or javascript. So you need to find another way to communicate a command to Google analytics. This is done by saving the campaign to session data (server side) with PHP or ASP. When the 301 redirect is executed, the recieving URL needs to check for session information and communicate it to Analytics. It should be communicated as campaign variables without using them in your url http://www.google.com/support/analytics/bin/answer.py?answer=27248
Good luck
September 6th, 2007 at 10:48 am
Hi Peter,
Thank you for a very interesting blog. I found it yesterday and have nearly read all your posts now :)
I have tried to apply your filter for Google Analytics so I can see what Adwords searchers really typed, and i have started seeing the first results.
But because the keyword data comes directly from the referrer, there is some URL encoded characters. In english I think the only problem is the spaces that are converted to “+”. But in Danish we have characters like æ, ø and å that are converted to “%C3%A6″, “%C3%B8″ and “%C3%A5″ and in Sweden and Germany there are different “problem characters”.
These characters make it hard to read the results. I have looked at the “find and replace” filter and tried letting it replace some of the characters, but it doesn’t seem to work.
Can you help me with this?
September 6th, 2007 at 12:29 pm
You can only fix this by running the referer data through a script on your page (PHP/ASP and Javascript can all do this) and passing it on to your favourite analytics variable. Variables like user defined of fake URL requests can then be read directly or filtered to show up in the adwords report.
I’m afraid I cannot help you any further, but a good programmer probably does this with great ease. Good luck!
November 27th, 2007 at 1:23 pm
Hi Peter
Can filters to include only certain sub-directories be viewed in our main analytics report or do we need to create separate reports and then set up the include filter for that report e.g. can I view a filter report for /products in our main site or do I need to create a Products report and set the filter up? (clear as mud!?)
November 27th, 2007 at 4:16 pm
The problem is that Google normally doesn’t allow for custom user segmentation. You can port specific variables to other ones and do some cross segmentation, but if you want the entire report specific for a group of visitors it is best to create a separate profile.
January 18th, 2008 at 12:29 am
Several days ago these filters stopped working properly. I’m sure you are aware of that. But do you have a solution?
January 18th, 2008 at 4:05 pm
Hi guys - great article. I’ve got a quick question. I want to filter Google natural search and Google blogsearch traffic out as seperate traffic streams. Does anyone know of a way to do this : I’ve not much hair left and that which I have is coming out by the handful in frustration.
Thanks in advance
Michael
April 14th, 2008 at 9:58 am
The ‘Exclude URL Query Parameters:’ dont seem to apply to these referal URL’s. Is it possible to add custom filters to grab whats needed or to exclude parameters?
April 17th, 2008 at 8:20 pm
Awesome Tips! I setup the filter, and tracked for a few days. After verifying 3 times with your website, I still couldn’t figure out why I wasn’t getting the correct stats. I simply wasn’t checking in the “User Defined” section of analytics. There is a whole new section I was completely unaware of, thanks a ton!
BJ
July 22nd, 2008 at 10:43 pm
@peter,
Full referrer URLs I just needed this filter. I was going crazy about the forums wich use the phpBB platform. You can never see the exact url where visitors came from.
But in most cases it’s major interesting to see what they wrote about you. :-)
Thanx…
Ps. how’s life?