Michael Roebuck's Blog

A client called recently and reported that the results for an A/B Ad Test they were running was out of whack. Google seemed to be favoring one ad over the other even though all the settings were set to show the ad equally.

After a quick call to Google, we find out that Google will no longer allow you to runs ads completely evenly. With the importance of Quality Score raised in the past months, that plays a factor into how often the ad shows. So the ad that Google deems more relevant to the keyword will get more play, even though we've selected it to run evenly over time.

Basically whichever ad is given the higher Quality Score in conjunction with a better CTR will appear more often despite having the setting to rotate evenly. It will still try to show them more evenly, but it most likely will give favor to the better performing ad.

I personally think this is a bad idea. We run A/B tests all the time to track effectiveness of ads and ensure we are always running the best possible ad. We also use this to track landing page effectiveness. By running the identical ad, we can track certain variables within a landing page.

Therefore, if any of you are using A/B tests for your Google ads, go check the data. It may start skewing very soon!

Labels: ab, google, test