Useful APIs for Your Web Site

Here’s a useful list of resources on how to create an API, compiled from posts that were published recently on this blog. The included APIs can provide a fantastic ways to enhance websites.

1. The WordPress(WP) API exposes a simple yet powerful interface to WP Query, the posts API, post meta API, users API, revisions API and many more. Chances are, if you can do it with WordPress, WP API will let you do it. [1] With this API you can get all posts with specific search term and display it on your website or get text from all posts and do text analytics.
Here is the link to post that is showing how to Retrieve Post Data Using the WordPress API with Python Script
You will find there python script that is able to get data from WordPress blog using WP API. This script will save downloaded data into csv file for further analysis or other purposes.

2. Everyone likes quotes. They can motivate, inspire or entertain. It is good to put quotes on website and here is the link to post that showing how to use 3 quotes API:
Quotes API for Web Designers and Developers
You will find there the source code in perl that will help to integrate quotes API from Random Famous Quotes, Forismatic.com and favqs.cominto into your website.

3. Fresh content is critical for many websites as it is keeping the users to return back. One of possible ways to have fresh content on website is adding news content to your website. Here is the set of posts where several free APIs such as Faroo API and Guardian APIs are shown how to use to get news feeds:
Getting the Data from the Web using PHP or Python for API
Getting Data from the Web with Perl and The Guardian API
Getting Data from the Web with Perl and Faroo API

In these posts different and most popular for web development programming languages (Perl, Python , PHP) are used with Faroo API and Guardian APIs to get fresh content.

4. Twitter API can be also used to put fresh content on web site as Twitter is increasingly being used for business or personal purposes. Additionally Twitter API is used as the source of data for data mining to find interesting information. Below is the post that is showing how to get data through Twitter API and how to process data.
Using Python for Mining Data From Twitter

5. The MediaWiki API is a web service that provides convenient access to Wikipedia. With a python module Wikipedia that wraps the MediaWiki API, you can focus on using Wikipedia data, not getting it. [2] That makes it easy to access and parse data from Wikipedia. Using this library you can search Wikipedia, get article summaries, get data like links and images from a page, and more.

This is a great way to complement the web site with Wikipedia information about web site product, service or topic discussed. The other example of usage could be showing to web users random page from Wikipedia, extracting topics or web links from Wikipedia content, tracking new pages or updates, using downloaded text in text mining projects. Here is the link to post with example how to use this API:
Getting Data From Wikipedia Using Python

References

1. WP REST API
2. Wikipedia API for Python



Quotes API for Web Designers and Developers

No one can deny the power of a good quote. They motivate and inspire us to be our best. [1]

Here are 3 quotes API that can be integrated in your website with source code example in perl.

1. Random Famous Quotes provides a random quote from famous movies in JSON format. This API is hosted on Mashape.
Mashape has example how to use API with unirest.post module however as perl is not supported by unirest here is the example of consuming API in different way. You need obtain Mashape key to run your own code.


#!/usr/bin/perl
print "Content-type: text/html\n\n";
use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );

my $ua = LWP::UserAgent->new;

$server_endpoint="https://andruxnet-random-famous-quotes.p.mashape.com/?cat=movies";
my $req = HTTP::Request->new(POST => $server_endpoint);   

$req->header('content-type' => 'application/x-www-form-urlencoded');
$req->header('X-Mashape-Key' => 'xxxxxxxxxxxxx');
$req->header('Accept' => 'application/json');

$req->content($post_data);

$resp = $ua->request($req);
if ($resp->is_success) {
    
     my $message = decode_json($resp->content);

print $message->{"quote"};
print $message->{"author"};

}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

2. Forismatic.com provides API to collection of the most inspiring expressions of mankind. The output is supported different formats such as xml, json, jsonp, html,
text. The quotes can be in English or Russian language. Below is the source code example how to consume this API. Note that in this example we use method GET while in previous example we used method POST. Also the code example is showing how to get output in json and html formats.



#!/usr/bin/perl
print "Content-type: text/html\n\n";
use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );


my $ua5 = LWP::UserAgent->new;

my $req5 = HTTP::Request->new(GET => "http://api.forismatic.com/api/1.0/?method=getQuote&format=json&lang=en");

$req5->header('content-type' => 'application/json');
$resp5 = $ua5->request($req5);

if ($resp5->is_success) {
          
     $message = decode_json($resp5->content);
     

print $message->{"quoteText"};
print $message->{"quoteAuthor"};

}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

# below is the code to get output in html format
my $req5 = HTTP::Request->new(GET => "http://api.forismatic.com/api/1.0/?method=getQuote&format=html&lang=en");

     $req5->header('content-type' => 'text/html');

     $resp5 = $ua5->request($req5);
     if ($resp5->is_success) {
    
     $message5 = $resp5->content;
     print $message5;
  }
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

3. favqs.com also provides quotes API. You can do many different things with quotes on this site on with API. FavQs allows you to collect, discover, and share your favorite quotes. You can get quote of the day, search by authors, tags or users. Here is the code example


#!/usr/bin/perl
print "Content-type: text/html\n\n";
use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );

    my $req5 = HTTP::Request->new(GET => "https://favqs.com/api/qotd");

     $req5->header('content-type' => 'application/json');
     $resp5 = $ua5->request($req5);
     if ($resp5->is_success) {
             
     $message5 = $resp5->content;
     print $message5;

     $message = decode_json($resp5->content);
   
print $message->{"quote"}->{"body"};
print $message->{"quote"}->{"author"};

  }
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

References

1. 38 of the Most Inspirational Leadership Quotes
2. LWP
3. How to send http get or post request in perl.html



Getting Data from the Web with Perl and The Guardian API

In one of previous post the Faroo API was used in order to get data content from the web. In this post we will look at different API that can be also used for downloading content from web. Here we will use the Guardian API / open platform.
At the time of writing at stated on website it has over 1.7 million pieces of content that can be used to build apps. This is the great opportunity to supplement your articles with related Guardian content. And we will look how to do this.

Specifically the perl script will be used for getting web search results with Guardian API. The following are the main steps in this perl script:
Connecting to Gurdian API
In this step we provide our API key and parameters to search function with the search terms string.


use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
my $ua = LWP::UserAgent->new;
my $server_endpoint = "http://content.guardianapis.com/search";
$server_endpoint=$server_endpoint."?q=$q&format=json&api-key=xxxxxxxx&page-size=10&page=$page";
my $req = HTTP::Request->new(GET => $server_endpoint);

Getting the Search Results and Decoding json data
In this step we decode json text that we got returned from our call to web search function.


use JSON qw( decode_json );
$resp = $ua->request($req);
if ($resp->is_success) {
        my $message = decode_json($resp->content);
###if we want to print to look at raw data:
###print $resp->content;
}

Displaying data
Now we are displaying data



use JSON qw( decode_json );

$items_N=10;
for ($i=0; $i<$items_N; $i++)
{

print  $message->{response}->{results}->[$i]->{webTitle};
print  $message->{response}->{results}->[$i]->{webUrl};
}

Conclusion
Thus we looked at how to connect to The Guardian API , how to get data returned by this API service, how to process json data and how to display data to user.
If your website is showing some content then it can be complemented by content returned from The Guardian API.
Feel free to ask questions, suggestions, modifications.

References
1. The Guardian
2. Online example of web search with the perl script

Below is the full perl source code


#!/usr/bin/perl

use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );
use CGI;

my $data = CGI->new();
my $q = $data->param('q');
my $start = $data->param('start');

if (($start eq "") || ($start == 0)) 
{$page=1;}
else
{  
    $page= int($start / 10) ;
}


if ($start eq "") {$start = 1;}

my $ua = LWP::UserAgent->new;


# http://open-platform.theguardian.com/documentation/search
my $server_endpoint = "http://content.guardianapis.com/search";
$server_endpoint=$server_endpoint."?q=$q&format=json&api-key=xxxx&page-size=10&page=$page";


my $req = HTTP::Request->new(GET => $server_endpoint);

$resp = $ua->request($req);
if ($resp->is_success) {
     
my $message = decode_json($resp->content);

###if we want to print to look at raw data:
###print $resp->content;

$items_N=10;
for ($i=0; $i<$items_N; $i++)
{
print  $message->{response}->{results}->[$i]->{webTitle};
print $message->{response}->{results}->[$i]->{webUrl};
}
  
}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}



Getting Data from the Web with Perl and Faroo API

As stated on Wikipedia “The number of available web APIs has grown consistently over the past years, as businesses realize the growth opportunities associated with running an open platform, that any developer can interact with.” [1]
For web developers web API (application programming interface) allows to create own application using existing functionality from another web application instead of creating everything from scratch.

For example if you are building application that is delivering information from the web to the users you can take Faroo API and you will need only to add user interface and connection to Faroo API service. Faroo API seems like a perfect solution for providing news api in all kind of format, either you are building a web application, website or mobile application.

This is because Faroo API is doing the work of getting data from the web. This API provides data from the web and has such functionalities as web search (more than 2 billion pages indexed as at the time of writing), news search (newspapers, magazines and blogs), trending news (grouped by topic, topics sorted by buzz), trending topics, trending terms, suggestions. The output of this API can be in different formats (json, xml, rss). You need only to make a call to Faroo API service and send the returned data to user interface. [2]

In this post the perl script that doing is showing data returned from web search for the given by user keywords will be implemented.

Connecting to Faroo API
The first step is to connect to Faroo API. The code snippet for this is shown below. The required parameters are specified via query string for server endpoint URL. For more details see Faroo API website [2]

  • q – search terms or keywords for web search
  • start – starting number of results
  • l – language, in our case english
  • src – source of data, in our case web search
  • f – format of returned data, in this example we use json format
  • key – registration key, should be obtained from Faroo API website for free

use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };

my $data = CGI->new();
my $q = $data->param('q');

my $ua = LWP::UserAgent->new;
my $server_endpoint = "http://www.faroo.com/api";
$server_endpoint=$server_endpoint."?q=$q&start=1&l=en&src=web&f=json&key=xxxxxxxxx&jsoncallback=?";
my $req = HTTP::Request->new(GET => $server_endpoint);
$resp = $ua->request($req);

Processing JSON Data and Displaying Data to Web User
If our call to Faroo API was successful we would get data and can start to display to web user as in the below code snippet:


use JSON qw( decode_json );

$resp = $ua->request($req);
if ($resp->is_success) {
   
my $message = decode_json($resp->content);

$items_N= $message->{count};
if($items_N >10) {$items_N=10;}
for ($i=0; $i<$items_N; $i++)
{

print  $message->{results}->[$i]->{title};

print  $message->{results}->[$i]->{url};

print  $message->{results}->[$i]->{kwic};

}

$next_number = 10 + $message->{start};    
}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

Full Source Code and Online Demo
The web search based on this perl code can be viewed and tested online at Demo for web search based on Faroo API

And here is the perl script, please note that some HTML formatting is not shown.


#!/usr/bin/perl
print "Content-type: text/html\n\n";
use LWP::UserAgent;
use HTTP::Request::Common qw{ POST };
use JSON qw( decode_json );
use CGI;


my $data = CGI->new();
my $q = $data->param('q');

my $ua = LWP::UserAgent->new;
my $server_endpoint = "http://www.faroo.com/api";
$server_endpoint=$server_endpoint."?q=$q&start=1&l=en&src=web&f=json&key=xxxxxxxx&jsoncallback=?";
my $req = HTTP::Request->new(GET => $server_endpoint);

$resp = $ua->request($req);
if ($resp->is_success) {
     print " SUCCESS...";
    
my $message = decode_json($resp->content);

$items_N= $message->{count};
if($items_N >10) {$items_N=10;}
for ($i=0; $i<$items_N; $i++)
{
print  $message->{results}->[$i]->{title};
print  $message->{results}->[$i]->{url};
print  $message->{results}->[$i]->{kwic};
}

$next_number = 10 + $message->{start};    
}
else {
    print "HTTP GET error code: ", $resp->code, "\n";
    print "HTTP GET error message: ", $resp->message, "\n";
}

Thus we looked at how to connect to Faroo API , how to get data returned by this API service, how to process json data and how to display data to user.
If your website is showing some content then it can be complemented by content returned from Faroo API.
Feel free to ask questions, suggestions, modifications.

References

1. Wikipedia

2. Faroo – Free Search API

3. Demo for web search based on Faroo API



How to Write to a Google Spreadsheet with a Perl Script

Google Docs is one of many cloud computing document-sharing services and provides an alternative to MS Office applications such as MS Excel. This post will show how to write data to Google Docs Spreadsheet and read data back from the same spreadsheet using perl programming language.

Google Sheets is an online spreadsheet application that lets you create and format spreadsheets and simultaneously work with other people. Here are the links for more information:
Overview of Google Sheets
Create and save a spreadsheet

First you need to create spreadsheet where the script will put some data. You need Google account for this. To create a new spreadsheet, go to your Google Drive , click the red ‘Create’ button, and select ‘Spreadsheet’ from the drop-down menu. As soon as you name the spreadsheet or start typing, Google Sheets will automatically save your work every few seconds.

At the top of the spreadsheet, you’ll see text that indicates when your spreadsheet was last saved. Now when you have spreadsheet you can call it in perl script by its name and write data to it. Perl module Net::Google::Spreadsheets provides all needed functions to do this. Below is the basic script that is opening spreadsheet, iterate through several rows and columns, multiply row index by column index and put it to the cell. Then it iterates again and read the values and print them to screen. Obviously you need to put real account information, spreadsheet and sheet names into this script to run it.

use Net::Google::Spreadsheets;
my $service = Net::Google::Spreadsheets->new( username => ‘xxxxxxx’, password => ‘xxxxxxx’ );
my @spreadsheets = $service->spreadsheets();
my $spreadsheet = $service->spreadsheet( { title => ‘spreadsheet_name’ } );
my $worksheet1 = $spreadsheet->worksheet( { title => ‘Sheet1’ } );

# get a cell
my $cell = $worksheet1->cell({col => 1, row => 5});
# update input value of a cell
$cell->input_value(‘new value’);
# print the value of cell
my $cell = $worksheet1->cell({col => 1, row => 5});
print “\n”;
print $cell->input_value;
print “\n”;
for($i=1; $i<4; $i++) { for($j=1; $j<4; $j++) { $cell = $worksheet1->cell({col => $i, row => $j}); $cell->input_value($i*$j);
} }
for($i=1; $i<4; $i++) { for($j=1; $j<4; $j++) { $cell = $worksheet1->cell({col => $i, row => $j});
print $cell->input_value;
print ” “;
}
print “\n”; }

To test that it really put values to spreadsheet you can access your spreadsheet at any time by opening your spreadsheet at Google Drive after you run perl script. So now we can create different programs that export or import data to spreadsheet.

References:

4 Free Alternatives To Microsoft Excel by Richard Wilson on May 8, 2013