Wednesday, July 15, 2015

If I HATE web lists so much - why do i click on them?!

I hate web lists.  I really do.  Especially the ones that list things ONE-AT-TIME so you have to load a page that is 90% ads. Irritating!  But, I saw one about "America's Top 100 Colleges". I work at a University - I wondered if we made the top 100.  So, I took the bait.

First I am sent to a page telling me what the list is about WITH LOTS OF ADS and a NEXT button.  I click NEXT. Ok, number 1, lots of ads another NEXT button.  This has already gotten old for me.  I was just curious what the top 100 colleges were - can't they just show me a list?!

So, I think to myself "I wonder if I could just harvest the list from the website with Perl.  I look at the URL - I see what they're doing - simply incrementing the folder by 1 each time.  Easy!  Next I look at the source code: TOO EASY!  They have only the STRONG tag once per page for the school name.  Cool!  So, two minutes of code and then: Tah-dah!

#!/usr/bin/perl -w
# Remember - this took two minutes, I know it's not perfect

use LWP::Simple;

my $rank=100;
my $base="";
while ($rank >0) {
    my $url = $base . "/$rank/";
    my $html = get($url);
    $html =~ m{<p><strong>(.*)</strong></p>}
          } or die "Cannot find school: $rank\n";
    $school = $1;
    print "$rank: $school\n";

I run it.  It doesn't work (I think).  It isn't changing the name.  Hmmm.  I keep killing it off before it finishes and examine the code.  I cannot see a problem so let it run - this time all the way to the end.  It does NOT change the name until it counts down to 50.  Wild!  I check the site - sure enough - there are no new schools after 50.  This is a list of the (alleged) Top 50 Colleges going under the name Top 100 Colleges.

Ha!  I guess they never thought anyone would ever have the patience to wade through all of those ads to see all 100 (I sure didn't).  This makes me wonder: Is it just a scam to get you to click through ads or did the content creator tell his boss he put all 100 pages up there knowing NO ONE would ever click through all of that.  If it's the latter - sorry buddy (or lady) for exposing you.

Tuesday, March 17, 2015

Switching Red Hat Linux to Oracle Linux

Normally, I'd rather fight than switch, but...

I recently decided to switch all of my Linux systems to Oracle Linux.  Why, you may ask?  Because I thought it was simpler.  Here's why:  

I was using Red Hat Enterprise Linux for some systems and CentOS for others and still Oracle Linux for Oracle systems.  I even had a couple Ubuntu servers.  I really like Red Hat flavored package management but I really preferred CentOS.  It tended to be easier to work with and have a lot more packages available in the repositories without adding any more.  

I used Red Hat Enterprise Linux in super critical places where the users demanded a corporate support contract.  Over time, I came to find it constraining and it was quite a bit different than the CentOS I was using.

As for Oracle Linux, I used it for Oracle (of course).  Oracle Linux had an RPM that setup the Oracle DBMS environment (including kernel tuning).  It was just a lot easier and was almost identical in feel to CentOS (which I was very used to).  And then I discovered ULN (Unbreakable Linux Network) and KSplice!  Now I could leave my systems up even when patching the kernel.  Amazing!

So now I could use Oracle Linux where I used CentOS (and pay nothing!) or use Oracle Linux and pay for the hypervisor and get all VMs free and get KSplice live kernel patching.  RHEL did not offer this and wanted to charge a separate license for each VM.  (Plus, Red Hat really didn't seem focused on the small higher ed market).

CentOS to Oracle Linux

To convert CentOS to Oracle Linux was really, really easy.  Oracle has shell script:

# curl -O 
# sh

Red Hat to Oracle Linux

To convert Red Hat, however, is quite a bit more steps but fairly easy as well. 

The first thing is to remove all of the Red Hat licensing. I shotgunned that with all of these commands:
# subscription-manager list --consumed
# subscription-manager remove --all
# subscription-manager config --rhsm.manage_repos=0
# subscription-manager list --consumed
# subscription-manager remove --serial=[insert serial number]

Next was a bit complicated since the default install did not have wget and was not in the default repository.  I used Perl and the Perl module LWP:

 #!/usr/bin/perl -w
use LWP::Simple;

This worked nicely, not having wget, and I was ready to edit the configuration files.  First, note the following changes in /etc/yum.repos.d/public-yum-ol6.repo.

name=Oracle Linux $releasever Update 6 installation media copy ($basearch)
enabled=1   <<==== changed from 0 to 1

Do the same for _latest and _add_ons.

Now edit  /etc/sysconfig/rhn/up2date and change /usr/share/rhn/RHNS-CA-CERT to sslCACert=/usr/share/rhn/ULN-CA-CERT.  Also, change serverURL setting from to

Now run:

# yum repolist
# yum list
# yum update
You will see:

[blah blah]...
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
 Userid: "Oracle OSS group (Open Source Software group) "
 From  : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
  redhat-logos.noarch 0:60.0.14-1.el6                                          


Now reboot and then login and check it out!

#  cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.6 (Santiago)

But we really know it's: 
#  cat /etc/oracle-release
Oracle Linux Server release 6.6


I really like EPEL so:

#  perl -MLWP::Simple -e "getstore('','epel-release-6-8.noarch.rpm');"
(feel free to use wget if you've got it)


Q: Will I have to relabel for SELinux?

No, I'm asking you.  Not sure, but so far so good so probably not or I'd have noticed an issue by now.


[Still trys to look to Red Hat but fails]
Loaded plugins: product-id, rhnplugin, subscription-manager
There was an error communicating with RHN.
RHN Satellite or RHN Classic support will be disabled.
Error communicating with server. The message was:
Sysid checksum is incorrect
Also, AFTER REBOOT, this annoying nag IF you didn't buy a subscription:

[root@oats ~]# yum update
Loaded plugins: product-id, rhnplugin, subscription-manager
There was an error communicating with ULN.
ULN support will be disabled.
Error communicating with server. The message was:
Sysid checksum is incorrect
Setting up Update Process
No Packages marked for Update


As you know, I like it.  I get to use Oracle Linux on the small one-offs for free and I have a half dozen systems that will enjoy KSplice - the rest get a basic subscription.  Well, that's my opinion (for now).

Monday, March 2, 2015

Book Review: Perl and XML

Recently, I pulled out my (mostly) unread copy of Perl and XML and rather than using it as a reference, began reading the entire thing starting with the Preface.  It was a very enjoyable read.  The fact that I enjoyed reading it so much was a surprise to me - who would've thought it possible?  I had worked with XML and RSS and LDIFs for a dozen years.  In the past, I basically just did a quick google and then thought "Yeah, there's the module I need" and installed it an ran with it.

This book explained the differences in the many modules and explained when and where to use them.  After nearly eight years I had an "Ah-hah!  That's why that program sometimes didn't always catch everything I was looking for!"  (I was using a generic XML parsing module when I should've been using XML::DOM

My only criticism (a minor one) is I would have liked to have seen more and varied examples in the RSS chapter.  But, it was an XML book, not a RSS book and the goal was not to show you the difference between Atom and iTunes in RSS feeds, etc.

If you use XML and Perl - you will want to READ this book.  Note:  I said READ this book.  Don't just buy it as a reference and put it on your shelf like I did (at first).  In fact, everyone reading this should buy it!  (It's only one penny on the used market at Why not?!)

Friday, June 14, 2013

YAPC::NA 2013 Austin, TX

Another great YAPC has come and gone.  What a great bunch of folks!  Although there were some great sessions, so far, my favorite was Mark Keating's keynote - specifically his advice to "buy this book".  I really didn't think I needed another Perl book - especially one that says "beginning" in the title.  But I looked at it anyway.  This is without a doubt the most complete "beginning" Perl book I've ever seen.  He covered things like the Perl community - which you don't see treated so extensively.  There was coverage of POD (which I love).  Of course it had whole chapter on OOP and one on creating modules - not typically what you'd call "beginning" Perl... And then I saw a chapter on Moose and had to buy it.  It looks to be fun summer reading for me.

Monday, December 10, 2012


I've recently starting trying to use POD for documenting more.  I love RCS so combining the two not only made sense but I found they compliment each other nicely.  I like that you can generate a man page or HTML from POD and you can use RCS variables to fill out part of the POD.  Here's my new file slurping template incorporating RCS and POD.

#!/usr/local/bin/perl -w


=head1 NAME 


=head1 VERSION







$in = shift;
open(IN,"<$in") or die "Cannot open $in: $!\n";
 # meow
 print $_;

Wednesday, March 7, 2012

Inspired by the new iPad ...

Okay, I can't take credit for it.  The Onion had this great story about the amazing power of the iPad.  That made me think - I wonder if you can could now use Perl on the new iPad from Apple.  I'm still not sure.  But, you can read several of your favorite books on the new Apple iPad.  I couldn't really find anyone scripting Perl on the new Apple iPad so not much more to say except to quote The Onion "new iPad, new iPad, new iPad, new iPad, new iPad, new iPad, new iPad, new iPad, new iPad, new iPad." 

Wednesday, December 21, 2011

Lost in Windowsland

For the first time in 15 years, (maybe ever*), I've been working on Windows servers.  I've always found Windows to be uncomfortable.  Lousy shell, few if any tools -and you can't script a GUI. But lately, I've been intrigued by Powershell and have been using it a bit for things.  With the addition of gVim, I am finding Windows to be more and more comfortable.  (gVim is a lifesaver!) 

Another thing I use is Strawberry Perl!  Actually, I use it to write Powershell scripts for large lists of users [add mailbox, delete mailbox, add/del mailContact, etc.]  But I am seeing some pretty cool Powershell tools (import-csv is cool) and someday, I'll just process files in Powershell and won't need Perl (on Windows).  Yeah right!!  And I'll switch to Notepad too.  Ha!

So, I'm pretty much weaned off of Cygwin - yeah, practically a Windows native now except for gVim and Perl.  I guess that okay, eh?

* I went directly from being the Netware guy to being the UNIX guy and skipped the Windows NT, etc.