Getting Data from the Web with Perl and The Guardian API

In one of previous post the Faroo API was used in order to get data content from the web. In this post we will look at different API that can be also used for downloading content from web. Here we will use the Guardian API / open platform.
At the time of writing at stated on website it has over 1.7 million pieces of content that can be used to build apps. This is the great opportunity to supplement your articles with related Guardian content. And we will look how to do this.

Specifically the perl script will be used for getting web search results with Guardian API. The following are the main steps in this perl script:
Connecting to Gurdian API
In this step we provide our API key and parameters to search function with the search terms string.

use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
my $ua = LWP::UserAgent->new;
my $server_endpoint = "http://content.guardianapis.com/search";
$server_endpoint=$server_endpoint."?q=$q&format=json&api-key=xxxxxxxx&page-size=10&page=$page";
my $req = HTTP::Request->new(GET => $server_endpoint);

Getting the Search Results and Decoding json data
In this step we decode json text that we got returned from our call to web search function.

use JSON qw( decode_json );
$resp = $ua->request($req);
if ($resp->is_success) {
        my $message = decode_json($resp->content);
###if we want to print to look at raw data:
###print $resp->content;
}

Displaying data
Now we are displaying data

use JSON qw( decode_json );

$items_N=10;
for ($i=0; $i<$items_N; $i++)
{

print  $message->{response}->{results}->[$i]->{webTitle};
print  $message->{response}->{results}->[$i]->{webUrl};
}

Conclusion
Thus we looked at how to connect to The Guardian API , how to get data returned by this API service, how to process json data and how to display data to user.
If your website is showing some content then it can be complemented by content returned from The Guardian API.
Feel free to ask questions, suggestions, modifications.

References
1. The Guardian
2. Online example of web search with the perl script

Below is the full perl source code

#!/usr/bin/perl

use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );
use CGI;

my $data = CGI->new();
my $q = $data->param('q');
my $start = $data->param('start');

if (($start eq "") || ($start == 0)) 
{$page=1;}
else
{  
    $page= int($start / 10) ;
}


if ($start eq "") {$start = 1;}

my $ua = LWP::UserAgent->new;


# http://open-platform.theguardian.com/documentation/search
my $server_endpoint = "http://content.guardianapis.com/search";
$server_endpoint=$server_endpoint."?q=$q&format=json&api-key=xxxx&page-size=10&page=$page";


my $req = HTTP::Request->new(GET => $server_endpoint);

$resp = $ua->request($req);
if ($resp->is_success) {
     
my $message = decode_json($resp->content);

###if we want to print to look at raw data:
###print $resp->content;

$items_N=10;
for ($i=0; $i<$items_N; $i++)
{
print  $message->{response}->{results}->[$i]->{webTitle};
print $message->{response}->{results}->[$i]->{webUrl};
}
  
}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}