How to perform SEO A/B testing in Google Search Console

For SEO strategists, it is sometimes difficult to know which of the many changes we make to our websites actually impact their overall SEO performance. Moving websites from page 10 to page 2 can usually be accomplished by following SEO best practices, but the trek to page 1? That requires far more granular attention to the specific changes we make to our landing pages. Enter SEO A/B testing, one of the best ways to narrow in and understand the effectiveness (or ineffectiveness) of specific optimizations.

Many digital marketers are comfortable using A/B testing features in their PPC campaigns or to analyze user behavior in Google Analytics, but fewer are incorporating this powerful strategy to better understand which of their on-page optimizations have the greatest impact on keyword rankings.

SEO A/B testing can seem intimidating, but with the right tools, it’s actually fairly simple to perform. Not only does A/B testing help SEOs identify the most impactful optimizations, but it also gives them a way to quantify their efforts. Although SEO A/B testing is more often utilized by advanced SEO strategists, site owners who are comfortable working on the backend of their website have a great opportunity to elevate their SEO strategy through well-structured A/B tests.

What is SEO A/B testing?

When it comes to controlled experiments, testing two variants is the fundamental building block from which all other testing is built. A/B testing is simply measuring how a single variant impacts an outcome. In the case of SEO, the outcomes are either better, worse, or static keyword rankings.

It doesn’t take much to get started. You’ll need Google Search Console for one (the absolute truth of your SEO rankings), as well as two clearly defined variants you want to test out. Lastly, site owners need to have a dev or staging environment where they can save a version of their website prior to the SEO A/B test just in case the changes do not produce the desired results.

Best use cases for SEO A/B testing

Anything can be A/B tested on our websites, but for SEO purposes, certain site elements are more likely to result in keyword rankings improvements because of the weight they carry in Google’s algorithm. For that reason, the below elements are the best use cases for SEO A/B tests.

Title tags

Choosing title tags is so important and has a huge impact on search results. Title tag changes are very impactful from a rankings perspective because they directly influence click-through-rate (CTR). Google has a normalized expected CTR for searches, and if your landing pages continually fall below the mark, it will negatively impact your overall chances of ranking.

Meta descriptions

For those websites that already have a lot of keywords on page one and are therefore getting lots of impressions, A/B testing meta descriptions can be really beneficial. Like title tags, they directly impact CTR, and improving them can result in significantly more clicks and thus better rankings. 

Schema markup

If you can, it’s good to add schema markup to all of your web pages (you miss all the shots you don’t take!), but if certain pages on your site still don’t have schema.org markup, adding it can be a great use case for an A/B test.

Internal links

Internal links communicate to Google site architecture, and they also distribute PageRank across our websites. Getting internal links right can produce dramatic keyword ranking improvements, particularly for larger websites with thousands of landing pages. Focus on header and footer links because of how much they shift PageRank. For websites with product pages, you can use A/B testing to find the best anchor text for your internal links.

New content

Adding good content to your landing pages is always beneficial because longer content implies topical depth. Using a landing page optimizer tool can help you improve the semantic richness of your content, and a subsequent A/B test empowers you to measure whether Google positively responds to those quality signals. 

Large groups of pages

For ecommerce sites or those who may add a large group of pages all at once, you can use A/B testing to measure whether those pages are crawled and indexed in a way that positively or negatively harms your existing rankings.

Information architecture

There are certain elements of information architecture that are more specific to SEO. Google likes page experience features like jumplinks and carousels, so understanding the impact of adding these features to your web pages is another reason to perform an SEO A/B Test.

Site migrations

Whenever you make core technical changes to your website, A/B testing is a great way to measure how those changes impact keyword rankings. It also helps prevent your site from experiencing a significant rankings drop in the long-term. 

The types of SEO A/B testing

There are a few different types of A/B tests that you can execute on your website. It will largely depend on the number of landing pages you have as well as the category of variants you are testing out.

A then B

The most basic form of A/B testing, this type of test will simply compare the performance of a single page with one different variant. This type of A/B test is better for smaller sites and easier to implement, particularly if you’re confident that the changes are in the right direction.  

Multi-page

This type of A/B test allows for far more statistically significant results and can be executed on large sites with hundreds to thousands of landing pages. Instead of measuring a single variant on a single page, pick two page groups of similar pages (I recommend at least 50 pages per cohort) and change the variant on all of those pages.  

Multivariate

This common form of experimentation has the same core mechanisms of an A/B test, but it increases the number of variants being tested. Multivariate testing can be great for measuring user behavior, but it is less effective in measuring search performance when you’re trying to discover whether a specific optimization has a direct impact on keyword rankings. 

How to perform an SEO A/B test 

The easiest way to do A/B tests is by using site snapshots and rollbacks. Basically, you take a site snapshot, make the change to your site, and sit back for a week and watch what happens. If the change to the site has improved your SEO, you’ll see it in your benchmarking. If you’ve made a mistake, then you just roll back to a previous version of your site and move forward with different optimizations. 

Here is a simple step-by-step process of an SEO A/B test:

1. Take a site snapshot or do a site backup prior to implementing changes so you can return to a previous version of your website if the change is not effective
2. Determine the variant being tested (for example, title tag, schema.org, and the others) and make the change to a page or cohort of similar pages
3. Wait 7-14 days to determine the impact of the change
4. Compare the keyword rankings for the variant page/s to the original. This can be done in Google Search Console or a Google Search Console Dashboard 

If the optimizations are impactful, you can proceed with making similar changes to other pages of your site. More targeted changes like adding keyword-rich title tags and meta descriptions won’t necessarily directly translate to other pages. However, more technical optimizations like schema.org and information architecture can be implemented across the entirety of your website with more confidence if an A/B test proves them impactful.

Some practical advice for SEO testing across hosting environments

The way you proceed with step one will depend on your dev and staging environment. If you’re hosting on DigitalOcean, you can take site snapshots. If you’re on WPEngine, you can choose a site backup to restore from. For the best in class, try Version Control from Git, which allows you to roll back to any version of your website. 

Version Control is like Track Changes on a Word Document, but the history never gets deleted. With Version Control, even if you delete something or change it, there is a chronology to all of the changes that have been made — when a line of code was created, when it was edited — and you can roll back at any time.

It’s also important to make sure all of the in-development pages and test environment pages on your site have the robots no index tag. Some SEO specialists might tell you that adding the pages to your blocklist in robots.txt or with on-page rel canonical tags would be sufficient, but at LinkGraph I’ve seen numerous examples where pages or subdomains were added to robots and continued to be in search results for months. Just adding a canonical tag is insufficient at blocking the dev domain from crawling and indexing.

The best strategy is to use the robots no index. The even better strategy is to use both robots no index and rel canonicals for added protection.

How long does it take to know whether the variant was effective?

How long you wait to measure your A/B tests will depend on how often Google crawls your website. If you test parts of your domain that aren’t crawled often, you may have to wait longer in order for Google to actually see your changes. If you’re changing primary pages that Google crawls frequently, you can likely see whether those optimizations had any impact within 7-14 days.

As you evaluate the effectiveness of your optimizations, beware of confounding variables. Multiple backlinks in a short period of time, adding Javascript, and of course, algorithm changes, can sharply impact keyword rankings. Any type of experimentation is more accurate when you can eliminate variables, so do your best to not schedule A/B tests during link building campaigns, core algorithm updates, or any period of high search volatility. 

When done correctly, A/B testing can be a powerful way to refine your SEO strategy toward maximum results. Not only can A/B testing help site owners make more data-driven decisions, but it can also help SEO strategists prove the value of their work to clients or executive leadership who may be wary of investing in SEO.

Manick Bhan is the founder and CTO of LinkGraph, an award-winning digital marketing and SEO agency that provides SEO, paid media, and content marketing services. He is also the founder and CEO of SearchAtlas, a software suite of free SEO tools. You can find Manick on Twitter @madmanick.