3.3. Perl XML-RPC Tutorial

The Wordtracker system can be accessed via XML-RPC.

The Wordtracker Web Service provided at xmlrpc.wordtracker.com has a dummied version at test.xmlrpc.wordtracker.com which allows you to play with the interface without having an account.

Each procedure in the API requires an access key. In the dummy server the access key is always "guest". All the results for the dummy version are fixed, although some parameters (eg the access key) are checked.

Connecting with the Frontier::RPC Client.

In order to connect to the XML-RPC API, you will need an XML-RPC client. For the examples below we use the one at http://search.cpan.org/dist/Frontier-RPC/. Download and install the Frontier::RPC2 module. You will also have to make sure the following modules are installed for Frontier::RPC2 to work properly:

Data-Dumper (in the Data directory)
MIME-Base64 (in the MIME directory)
MD5 (in the MD5 directory)
HTML-Tagset (in the HTML directory)
HTML-Parser (in the HTML directory)
URI (in the URI directory)
libnet (in the Net directory)
libwww-perl (in the HTTP directory)
XML-Parser (in the XML directory)

To get up and running quickly, create the file crude_ping.php as follows:

#!/usr/bin/perl -w

use strict;
use Frontier::Client;

my $server_url = 'http://test.xmlrpc.wordtracker.com';
my $server = Frontier::Client->new('url' => $server_url);

my $result = $server->call('ping', 'guest');

if (${$result} == 1) {
    print "Successfully pinged test account";
    exit 0;
}
                

This should produce the output:

Successfully pinged guest account.

The first thing this code does is create a client pointing at the dummy version of the service. Ping is a special procedure that confirms that your account is active. Here, because we are using the guest account, it will always return success if the access key used is "guest". For the Frontier::RPC client, we pass the access key in the query method.

In practice there are several things that can go wrong.

If you receive a bad hostname error...

500 can't connect to test.xmlrpc.wordtrackerx.com
(bad hostname 'test.xmlrpc.wordtrackerx.com')
                

...this is most likely due to an incorrect url.

If you receive a server error...

Fault returned from XML RPC Server, fault code -32601: server error.
requested method pingx does not exist.
                

...this is most likely due to an incorrect method e.g. pingx instead of ping.

If you receive this server error...

Fault returned from XML RPC Server, 
fault code -1: Unknown access key
                

...this is most likely due to an incorrect username e.g. guestx instead of guest.

We are now ready to tackle a more useful procedure...

Simulating a Wordtracker Search.

A typical call to Wordtracker is to find the search popularity of a selection of keyphrases. This is achieved with one of get_exact_phrase_popularity, get_embedded_phrase_popularity and get_all_words_popularity. The simplest is get_exact_phrase_popularity, which will search the database for phrases which people have typed into metacrawlers. They can be retrieved exactly with get_exact_phrase_popularity.

The following example uses the dummy version of get_exact_phrase_popularity to demonstrate the general approach. It omits error checking to reduce clutter.

#!/usr/bin/perl -w

use strict;
use Frontier::Client;

my $server_url = 'http://test.xmlrpc.wordtracker.com';
my $server = Frontier::Client->new('url' => $server_url);

my $result = $server->call(
            'get_exact_phrase_popularity',
            'guest',
            ['mp3', 'britney spears'],
            'case_distinct',
            0,
            1,
            'exclude_adult',
            100,
            10);

foreach my $key ( sort keys %{$result} ) {
    print "$key = $result->{$key}\n";
}
                

We create the client as before, but this time we need to pass in rather more parameters.

The dummy version of the Wordtracker API does actually perform some basic parameter checks. If the case parameter is not one of the Strings "case_distinct", "case_folded", or "case_sensitive", an error is returned. Similarly, the adult parameter must be one of "adult_only", "include_adult", and "exclude_adult".

The dummy version of the interface is simplistic in that it returns a count of 999 for each phrase queried. As an aside, although this dummied procedure returns a single result for each keyphrase, we will see that the real system could produce multiple results.

When run, the sample should output...

mp3 = 999
britney spears = 999
                

Fetching Some Real Keyphrase Data

The above examples have all used a stubbed version of the web services. Once you have a Wordtracker web services account you will be sent an access key. This will allow you to run the following sample against the live server.

#!/usr/bin/perl -w

use strict;
use Frontier::Client;

my $server_url = 'http://xmlrpc.wordtracker.com';
my $server = Frontier::Client->new('url' => $server_url);

my $result = $server->call(
            'get_exact_phrase_popularity',
            'guest', 			# Replace with access key
            ['mp3', 'Britney Spears'],
            'case_distinct',
            0,
            1,
            'exclude_adult',
            100,
            10);

foreach my $key ( sort keys %{$result} ) {
    print "$key = $result->{$key}\n";
}

                

The first thing we do to rework the stub example is change the URL to the live server at http://xmlrpc.wordtracker.com/. The string "guest" in the above code must be changed to your actual access key. You must have access to the get_exact_phrase_popularity procedure from your account to be able to run this example, if not you will have to experiment with one that you do have.

This example demonstrates how to unpack an XML-RPC struct returned from a call into its components. Here is the output.

mp3 = 16734
britney spears = 14245
Britney Spears = 4100
MP3 = 2040
mp3s = 1263
Mp3s = 574
Mp3 = 556
Britney spears = 330
BRITNEY SPEARS = 276
MP3s = 90
britney Spears = 50
MP3S = 12
Britney SPears = 11
mP3 = 6
bRITNEY sPEARS = 6
BRITNEY Spears = 4
britney Spearss = 2
britney spearss = 2
BRitney SPears = 2
BRitney spears = 2
                

That's it, you're done.