Friday, June 8, 2012

Retrieve Windows System Information with Perl


For anyone that has to troubleshoot Windows systems, being able to quickly and easily identify information about the computer such as OS info, drive information, network settings, and so on, can be a real time saver.  This Perl script makes use of several Win32 modules to determine information about the installed operating system, the computer name, processor information, RAM information, drive information, and computer network settings.  When running the script, you may want to take advantage of redirection to save the output to a file since it can be a bit on the lengthy side if the computer has multiple processors, multiple network adapters, etc.  To take advantage of redirection run the script as follows:

perl WinInfo.pl > output.txt

This will save the output of the script to a file called output.txt. 

use Win32;
use Win32::SystemInfo;
use Win32::DriveInfo;
use Win32::IPConfig;
use strict;
use warnings;

print "OS Information\n";
my $computer=Win32::NodeName();
print "The computer name is $computer\n";

my $domain=Win32::DomainName();
print "The computer is a member of the $domain domain/workgroup\n";

my $OS=Win32::GetOSDisplayName();
print "The OS is $OS\n";

my $fs=Win32::FsType();
print "The filesytem is $fs\n";

my $user=Win32::LoginName();
print "The current user is $user\n";

my $admin=Win32::IsAdminUser();
if($admin!=0){
    print "$user is running this script as admin\n\n\n";
}
else{
    print "$user is not running this script as admin\n\n\n";
}

print "Processor and RAM Information\n";
my %processor;
Win32::SystemInfo::ProcessorInfo(%processor);
for (my $i=0;$i<$processor{NumProcessors};$i++) {
    print "Processor$i\n";
    print "Processor Name: " . $processor{"Processor$i"}{ProcessorName} . "\n";
    print "Processor Info: " . $processor{"Processor$i"}{Identifier} . "\n";
    print "Processor Speed: " . $processor{"Processor$i"}{MHZ} . "MHz\n\n";
}

my %memory;
Win32::SystemInfo::MemoryStatus(%memory, 'GB');
print "The computer has $memory{TotalPhys} GB of RAM\n\n\n";

my %dtypes=(0 => "Undertmined",
1 => "Does Not Exist",
2 => "Removable",
3 => "Hardrive",
4 => "Network",
5 => "CDROM",
6 => "RAM Disk");

print "Drive Information\n";
my @drives = Win32::DriveInfo::DrivesInUse();
foreach my $drive (@drives){
    my $type=Win32::DriveInfo::DriveType($drive);
    print "Drive $drive is a $dtypes{$type}\n";
  
}

print "\n\nNetwork Information";
my $ipconfig = Win32::IPConfig->new($computer)
        or die "Unable to connect to $computer\n";
foreach my $adapter ($ipconfig->get_adapters) {
    print "\nAdapter '", $adapter->get_name, "':\n";

    print "Description=", $adapter->get_description, "\n";

    print "DHCP enabled=",
    $adapter->is_dhcp_enabled ? "Yes" : "No", "\n";

    my @ipaddresses = $adapter->get_ipaddresses;
    print "IP addresses=@ipaddresses (", scalar @ipaddresses, ")\n";

    my @subnet_masks = $adapter->get_subnet_masks;
    print "subnet masks=@subnet_masks (", scalar @subnet_masks, ")\n";

    my @gateways = $adapter->get_gateways;
    print "gateways=@gateways (", scalar @gateways, ")\n";

    print "domain=", $adapter->get_domain, "\n";

    my @dns = $adapter->get_dns;
    print "dns=@dns (", scalar @dns, ")\n";

    my @wins = $adapter->get_wins;
    print "wins=@wins (", scalar @wins, ")\n";
}

Kobo has over 2 million ebooks to choose from!

Wednesday, June 6, 2012

Using Data::Validate Modules to Validate Application Data


In an earlier approach I provided A Brief Introduction to Input Validation where the basic concepts of whitelisting and blacklisting were introduced to readers.  While every programmer should be familiar with the idea of writing input validation filters it can often be challenging to develop a filter that minimizes both the numbers of false positives as well as the number of false negatives, and thus it often pays to use well tested routines for input validation where feasible.   CPAN contains many such routines that can usually be located by searching CPAN for terms like Valid, Validate, Validator, and Validation.  One CPAN namespace in particular that has many useful routines is the Data::Validate namespace, which contains routines for validating numbers, IP addresses, email addresses, URLs and more.  In the code snippet below we will take a look at the Data::Validate::IP (http://search.cpan.org/~neely/Data-Validate-IP-0.14/lib/Data/Validate/IP.pm) and the Data::Validate::Email (http://search.cpan.org/~sonnen/Data-Validate-Email-0.04/Email.pm) Perl modules.  

 #!usr/bin/perl

use strict;
use warnings;

use Data::Validate::IP qw(is_ipv4);

my @IPs=('192.168.1.5','127.0.0.1','116.12.54.257','abc.123.123.123');
foreach my $IP (@IPs){
    if(is_ipv4($IP)){
        print "$IP is a valid ipv4 address\n";
    }
    else{
        print "$IP is invalid\n";
    }
}

use Data::Validate::Email qw(is_email);
my @Emails=('someone@example.com','someone&example.net','someone@example.123','someone@example.info');
foreach my $Email (@Emails){
    if(is_email($Email)){
        print "$Email is a valid Email address\n";
    }
    else{
        print "$Email is invalid\n";
    }
}

 
Once you select a validation method for your application it is also imperative that you take the time to test your validation routine to make sure that valid inputs are properly passed on to the application and that any invalid inputs are flagged as such.  Such testing should involve both use cases as well as “abuse” cases designed to try to bypass the filter.  The need for testing is especially true if you decide to implement your own validation routine as it is not always easy to think off all the possible ways somewhat could attempt to format an input string.  Remember that input validation is typically an application’s front line defense in that it exists to minimize the risk of improper or malicious data from ever being processed by the application. 

Searching Pubmed Using Regular Expression Based Pattern Matching



This is a script that I wrote a number of years ago that provides a way of searching Pubmed abstracts using regular expression based pattern matching and, hence, it provides a way to pull out mutation data and other types of biomedical data that can be fit to a text pattern but not easily represented with keywords.  Publications describing some sample bioinformatics usages of the script for mining mutation data, gathering links to biomedical resources, and profiling developmental factors that play a role in the inner ear can be found in the links below.  I am reposting this code here because it seems the original host (bioinformatics.org) no longer makes a copy of the script available and I have recently received a number of requests for copies of the script. 

Sample Publications:


 
Code: 
 
#!/usr/bin/perl

# PREP (Perl RegExps for Pubmed) is a script that allows the use of 
# Perl regexs in the searching of Pubmed records, providing the ability to search 
# records for textual patterns as well as keywords

# Copyright 2005- Christopher M. Frenz
# This script is free sofware it may be used, copied, redistributed, and/or modified
# under the terms laid forth in the Perl Artisic License 

# Please cite this script in any publication in which literature cited within the
# publication was located using the PREP.pl script.  

# Usage: perl PREPv1-0.pl PubmedQueryTerms

# Usage of this script requires the LWP and XML::LibXML modules are installed
use LWP;
use XML::LibXML; #Version 1.58 used for development and testing

# Change the variable below to set the text pattern that Perl 
# will seek to match in the returned results
my $regex='[ARNDCEQGHILKMFPSTWYV]\d+[ARNDCEQGHILKMFPSTWYV]';

my $request;
my $response;
my $query;

# Concatenates arguments passed to script to form Pubmed query
$query=join(" ", @ARGV);

# Creates the URL to search Pubmed
my $baseurl="http://www.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?";
my $url=$baseurl . "db=Pubmed&retmax=1&usehistory=y&term=" . $query;


# Searches Pubmed and Returns the number of results
# as well as the session information needed for results retrieval
$request=LWP::UserAgent->new();
$response=$request->get($url);
my $results= $response->content;
die unless $response->is_success;
print "PubMed Search Results \n";
$results=~/<Count>(\d+)<\/Count>/;
   my $NumAbstracts=$1;
$results=~/<QueryKey>(\d+)<\/QueryKey>/;
   my $QueryKey=$1;
$results=~/<WebEnv>(.*?)<\/WebEnv>/;
   my $WebEnv=$1;
print "$NumAbstracts are Available \n";
print "Query Key= $QueryKey \n";
print "WebEnv= $WebEnv \n";

# Opens a file for output
open(OFile, ">PREPout.html");

my $parser=XML::LibXML->new;

my $retmax=500; #Number of records to be retrieved per request-Max 500
my $retstart=0; #Record number to start retreival from

# Creates the URL needed to retrieve results
$baseurl="http://www.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?";
my $url2="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=";

my $Count=0;
# Retreives results in XML format
for($retstart=0;$retstart<=$NumAbstracts;$retstart+=$retmax){
   print "Processing record # $retstart \n";
   $url=$baseurl . "rettype=abstract&retmode=xml&retstart=$retstart&retmax=$retmax&db=Pubmed&query_key=$QueryKey&WebEnv=$WebEnv";

   $response=$request->get($url);
   $results=$response->content;
   die unless $response->is_success;

   # Uses a DOM based XML parser to process returned results
   my $domtree=$parser->parse_string($results);
   @Records=$domtree->getElementsByTagName("PubmedArticle"); 
   my $i=0;
   foreach(@Records){
# Extracts element data for regex processing and output formatting
      $titles=$Records[$i]->getElementsByTagName("ArticleTitle");
      $journals=$Records[$i]->getElementsByTagName("MedlineTA");
      $volumes=$Records[$i]->getElementsByTagName("Volume");
      $pgnums=$Records[$i]->getElementsByTagName("MedlinePgn");
      $abstracts=$Records[$i]->getElementsByTagName("AbstractText");
      $IDS=$Records[$i]->getElementsByTagName("PMID");


       # Processes title and abstract for pattern match and if a match occurs
       # data is written to output
       if($titles=~/($regex)/ or $abstracts=~/($regex)/){
           print OFile "<h1>Pattern Match: $1 </h1>\n";
           print OFile "<h3><a href=\"$url2$IDS\">$titles </a></h3> \n";
           print OFile "<p>$journals $volumes, $pgnums </p>\n";
           print OFile "<p>$abstracts </p>\n\n";
           $Count=$Count+1;
       }
       $i=$i+1;
   }
}
close OFile;
print "$Count records matched the pattern";

Tuesday, June 5, 2012

Improving the Image of Perl (Perl Marketing)


One of the comments to a posting I made about Scientific Computing in Perl raised the important question “If you'd like to have more people use Perl for scientific computation, it might be a good idea to think about and discuss what might have caused the shift of some people towards Python?”  I think this is a great question and I think it applies to more than just the scientific computing aspects of Perl, but Perl in general.  I think Perl has an image problem and because of this it is no longer trendy to use Perl.  If you talk to anyone that is not a Perl programmer about Perl, you tend to hear the same unfounded complaints over and over – e.g. Perl has a hard to understand and maintain syntax, OOP in Perl seems like an afterthought, etc.  Many such comments are unfounded or have long been addressed, but awareness of this does not seem to reach those that are not already working with Perl.  In many cases, I wouldn’t even be surprised if those making the complaints didn’t even have any true firsthand experience with Perl. 

While it is certainly possible to write some highly obfuscated Perl code, this is not the way it has to be.  A Perl programmer or corporation that employs them could easily make use of modules like Perl::Tidy and Perl::Critic to help maintain readability as well as having coding standards in place.  In fact many in the Perl community would actually strongly support such practices.   In terms of OOP support Perl now has Moose, which provides a more modern object system for Perl applications.  Perl has also produced a number of modern Web development frameworks, such as Dancer and Mojolicious.  Perl is even being used for “Big Data” analytics problems (http://blogs.perl.org/users/jt_smith/2012/05/perl-for-big-data.html).  Perl has been anything but a stagnant language and clearly still has a strong and innovative community supporting it.  One just has to look at a Perl news aggregation like the Perl Weekly, to see all of the interesting things members of the Perl community are working on and discussing. 



This therefore raises the question of how we can go about improving the marketing of Perl, to make the advancements and growing utility of Perl more widely known to those outside of the Perl community.  Here are some possibilities to consider:

1)     Writing articles, blog posts, comments, etc that highlight the utility of Perl and the recent advancements of Perl.  In particular I think articles in programming, Linux, and open source publications would be the most effective, since these are read by more than those with a vested interest in Perl. 
2)     When coming across an article or blog post that contains a discussion about something really interesting in Perl take the time to link to it, “Like” it, “+1” it, post to reddit, Hacker News, etc.  This could help to further generate a buzz about some of the interesting things the Perl community is working on. 
3)     Anybody remember Perlcast (http://perlcast.com/)?  It might be interesting to revive something like this as another way of spreading Perl news, particularly if it could be incorporated into one or more sites that promote a large number of technology podcasts like IT Conversations as this would get the podcasts in front of even more people.  It was a long time ago now, but if anyone is interested here is a link to my Perlcast interview - http://www.perlcast.com/audio/Perlcast_Interview_012_Frenz.mp3
 
4)     Many cities have Perl Mongers groups that give Perl related talks and tutorials.  Encourage those in your group to video such talks and post them online. 
5)     Highlight major projects, programs, and Websites that are built with Perl.  Slashdot, DuckDuckGo, Blekko, etc. 
6)     Perl outreach – volunteer to give a free lecture or class(es) on Perl at a local college, high school, library, etc.  The long-term future of Perl probably relies more on its popularity with and use by the next generation of programmers more so than it does with established players. 
7)     I’m sure there are plenty more, but these would provide a good starting point. 

Kobo has over 2 million ebooks to choose from!

Monday, June 4, 2012

Scientific Computing in Perl


I come from a scientific computing background (computational biology) and as such have often had to perform numerical computing tasks in numerous programming languages ranging from tried and true stalwarts like Fortran and C to languages like MATLAB.  A large amount of the scientific computing work I have done has also been done in Perl which has a long and rich history in the field of bioinformatics.  Recently, however, there seems to be a shift in the trend of people entering computational biology and bioinformatics to tend towards learning and using Python instead of Perl, citing the availability of libraries such as SciPy (for numerical computing) and interfaces such as RPy (for statistical computing with R).  This post is not meant to start a flame war with any fans of Python or Python’s scientific computing capabilities, as the Python community has done some great work in the area of scientific computing and has even produced some tools I have made extensive use of, such as Pymol (http://www.pymol.org/).  In fact I actually think competition is always a good thing.  Rather, I intend to use this post to raise some awareness of some of the Perl modules that can be of great benefit to anyone considering using Perl for scientific computing purposes. 

Perl Data Language (PDL) – allows Perl to manipulate large n-dimensional arrays (similar to MATLAB, NumPy, etc) in a quick and efficient manner.  The PDL namespace on CPAN contains many modules that should be of interest to scientific programmers including interfaces for numerical computing functions found in the GNU Scientific Library (http://www.gnu.org/software/gsl/).  More information on PDL can be found at http://pdl.perl.org/.

The Statistics namespace on CPAN – This namespace includes many Perl modules that allow the computation of numerous statistical analyses, ranging from basic descriptive statistics (Statistics::Descriptive) to more sophisticated analyses like multivariate regression (e.g.  Statistics::Regression).  There is even a module that allows your Perl code to interface with the statistical computing language R (Statistics::R). 

The Math namespace on CPAN – This namespace includes all kinds of advanced math functions that can be easily integrated into a Perl application.  While there are too many to mention in such a short synopsis, modules located here will provide support for dealing with areas of math such as trigonometry (Math::Trig) and complex numbers (Math::Complex) as well as provide access to algorithms for solving many types of math problems, such as solving for the roots of polynomial equations (Math::Polynomial::Solve). 

BioPerl – while not as generic to scientific computing as some of the ones mentioned above, BioPerl is a large and widely used collection of Perl modules for performing bioinformatics tasks.  More information on BioPerl can be found at http://www.bioperl.org/wiki/Main_Page. 

Perl’s Inline capability – the ability to integrate code from other languages into your Perl application (particularly inline C) can help to give your applications the flexibility and ease of use of Perl while allowing you to optimize certain parts of your application to improve performance.  

Of course there are many other modules as well that could be of great use for many scientific applications, as there are also modules that deal with data mining techniques, machine learning, and numerous other aspects of data analysis.  The intent, however, was to demonstrate that Perl does have a rich set of tools available to it for use in the development of scientific computing applications and that Perl should not be quickly dismissed in favor of Python.  I think for anyone entering the fields of computational biology or bioinformatics, Perl is still a language worth learning and still my preferred language for bioinformatics tasks.  Even if you decide in favor of Python for your new projects, you should be aware that many existing projects are written in Perl, and you may well have to maintain, modify, or interface with such codebases, where knowledge of Perl will only be to your advantage. 

Kobo has over 2 million ebooks to choose from!