Although keyword research is a wonderful thing, it’s not perfect. Mark Nunney looks at the limits of traditional keyword research and how you can go beyond those restrictions to a reliable scientific process.

Keyword research is just research

With traditional keyword research we first explore the Popularity and Competition of keywords we might want to target with SEO and PPC. Let’s look at each of those:

• Popularity is how often a keyword is searched with

• Competition is a measure of how hard it is to beat other websites competing for a keyword

Popularity and Competition can then be combined into a single metric, often called KEI (Keyword Effectiveness Index).

So traditional keyword research helps make sure your search engine optimization (SEO) efforts target keywords that have sufficient popularity and offer the chance of beating the competition.

The image below shows us the results of search of Wordtracker’s Keywords research tool with the keyword gourmet chocolate.

Above, for the first 10 of 93 keywords containing gourmet chocolate, we see:

• Searches An estimate of the number of searches made in one month with keywords containing gourmet chocolate.

• Competition (IAAT) A rating of the level of serious competition (the higher, the tougher).

• KEI1 and KEI3 Two different ways of calculating KEI (Keyword Effectiveness Index) to suggest keywords with an attractive ratio of Popularity and Competition.

• Google Count The number of pages competing on Google.

That’s great. Indeed it’s amazing and perhaps the best value for money market research you can get. (You can take Wordtracker Keywords tool for a free trial here.) But we must be aware of its limitations, including the following:

• All keyword research tools are subject to error so you must verify their data before significantly investing in their results.

• Keyword research tools’ data is from samples of real searches that are at best an average of all searchers in the country the data comes from. But is it the country you are targeting? Are you targeting the whole country or a region within it? Are you targeting a niche market that can’t be represented by a sample of all searchers?

• The competition data might be powerful like Wordtracker’s but it’s not telling you how your site compares to the competition. It is not a relative judgement. Competition might be plenty and tough but you might already be tougher (in your market).

• You don’t know how well your site will perform against the competition and therefore how hard it will be to get non-paid-for (organic) visits from those searching with a particular keyword.

• If you get the visitors, can your site convert them into sales or some other wanted response?

Here are two ways to deal with these limitations ...

1) Verify keyword popularity and response rates with PPC

With pay per click (PPC) advertising you can verify a keyword’s popularity (how often it is searched with) for specific countries and regions. You can also find out if your site can convert visitors who have searched with it.

Let’s use the example of targeting the keyword gourmet chocolate recipes. Keyword research suggests there are at least a few thousand searches made each month with gourmet chocolate recipes and that the competition looks beatable.

With Google Adwords (PPC advertising) you can bid for the keyword gourmet chocolate recipes and have an advert for your site appear on Google’s results pages when searches are made with gourmet chocolate recipes.

Your Google Adwords account will tell you how many times your ad appeared when gourmet chocolate recipes was searched with. From this you will get a good idea about the keyword’s true popularity.

Of course, you don’t care about visits if they don’t convert to sales. If your ad interests searchers enough for them to click on it and visit your site then you will also find out if you can convert them to sales (or whatever response you hope to achieve).

2) Invest in SEO for keywords you know you can beat the competition for

PPC can verify a keyword’s Popularity and the ability of your site to convert those searching with it. But it can’t tell you if you can beat the competition for organic results from search engines.

And for our gourmet chocolates recipes example, the competition is 625,000 other website pages. See image:

Even with the best competition metrics from Wordtracker Keywords tool, you are only making an informed guess about how hard it is for your site to beat the competition. That is until you have actually done so.

So you must go into SEO battle on the organic results against those 625,000 other pages.

As soon as your site gets just a small amount of organic search engine traffic from a keyword then you know you can beat the competition to some degree. (I’ll assume PPC testing has shown you can convert these visitors.)

You can now build on any small successes by targeting those keywords with more SEO.

So with SEO you first target a range of keywords that PPC has proven to be responsive. And you wait for results, investing in those keywords that deliver.

You’ll have moved from targeting keywords that might bring results to targeting keywords you have proven you can beat the competition for and get response from.

You’ll have moved from a craft that provides intelligent guesswork to a scientific method based on proven success.

More than the traditional keyword research process

We can divide the keyword research process into two distinct phases:

1) The Keyword Research Funnel is for sites without existing organic (non-paid) search engine traffic.

2) The Keyword Research Circle of Response is for sites with organic search engine traffic.

If your site is new then you must start with the Keyword Research Funnel. When your site has consistent organic traffic from search engines you can move on to the Keyword Research Circle of Response.

Sites with existing organic search engine traffic can jump straight into the Circle of Response.

The whole process is captured in the process diagram below.

In Wordtracker Keyword Research Masterclass - How to deliver visits, response and profits to your website I show you how to use this process, one step at a time. You’ll also learn how to use Wordtracker Keywords to save a lot of time.

About Mark Nunney

Mark Nunney has been a successful professional SEO since 2000. He is CEO of The Website Marketing Company and he publishes Leadership & Management Review from ThinkingManagers.com, the business management website.

Mark wrote SEO for Profit, Wordtracker Masterclass: Keyword Research book and co-wrote Wordtracker Masterclass: Link Building with Ken McGaffin. He is also the founder and project manager of Wordtracker Strategizer.

You can follow Mark Nunney's SEO on Twitter, LinkedIn and Google+ and read a Q&A here.