I hate web lists. I really do. Especially the ones that list things ONE-AT-TIME so you have to load a page that is 90% ads. Irritating! But, I saw one about "America's Top 100 Colleges". I work at a University - I wondered if we made the top 100. So, I took the bait.
First I am sent to a page telling me what the list is about WITH LOTS OF ADS and a NEXT button. I click NEXT. Ok, number 1, lots of ads another NEXT button. This has already gotten old for me. I was just curious what the top 100 colleges were - can't they just show me a list?!
So, I think to myself "I wonder if I could just harvest the list from the website with Perl. I look at the URL - I see what they're doing - simply incrementing the folder by 1 each time. Easy! Next I look at the source code: TOO EASY! They have only the STRONG tag once per page for the school name. Cool! So, two minutes of code and then: Tah-dah!
#!/usr/bin/perl -w
# Remember - this took two minutes, I know it's not perfect
use LWP::Simple;
my $rank=100;
my $base="http://www3.forbes.com/forbeswoman/americas-top-100-colleges/";
while ($rank >0) {
my $url = $base . "/$rank/";
my $html = get($url);
$html =~ m{<p><strong>(.*)</strong></p>}
} or die "Cannot find school: $rank\n";
$school = $1;
print "$rank: $school\n";
$rank--;
}
I run it. It doesn't work (I think). It isn't changing the name. Hmmm. I keep killing it off before it finishes and examine the code. I cannot see a problem so let it run - this time all the way to the end. It does NOT change the name until it counts down to 50. Wild! I check the site - sure enough - there are no new schools after 50. This is a list of the (alleged) Top 50 Colleges going under the name Top 100 Colleges.
Ha! I guess they never thought anyone would ever have the patience to wade through all of those ads to see all 100 (I sure didn't). This makes me wonder: Is it just a scam to get you to click through ads or did the content creator tell his boss he put all 100 pages up there knowing NO ONE would ever click through all of that. If it's the latter - sorry buddy (or lady) for exposing you.
Wednesday, July 15, 2015
Tuesday, March 17, 2015
Switching Red Hat Linux to Oracle Linux
Normally, I'd rather fight than switch, but...
I recently decided to switch all of my Linux systems to Oracle Linux. Why, you may ask? Because I thought it was simpler. Here's why:I was using Red Hat Enterprise Linux for some systems and CentOS for others and still Oracle Linux for Oracle systems. I even had a couple Ubuntu servers. I really like Red Hat flavored package management but I really preferred CentOS. It tended to be easier to work with and have a lot more packages available in the repositories without adding any more.
I used Red Hat Enterprise Linux in super critical places where the users demanded a corporate support contract. Over time, I came to find it constraining and it was quite a bit different than the CentOS I was using.
As for Oracle Linux, I used it for Oracle (of course). Oracle Linux had an RPM that setup the Oracle DBMS environment (including kernel tuning). It was just a lot easier and was almost identical in feel to CentOS (which I was very used to). And then I discovered ULN (Unbreakable Linux Network) and KSplice! Now I could leave my systems up even when patching the kernel. Amazing!
So now I could use Oracle Linux where I used CentOS (and pay nothing!) or use Oracle Linux and pay for the hypervisor and get all VMs free and get KSplice live kernel patching. RHEL did not offer this and wanted to charge a separate license for each VM. (Plus, Red Hat really didn't seem focused on the small higher ed market).
CentOS to Oracle Linux
To convert CentOS to Oracle Linux was really, really easy. Oracle has shell script: centos2ol.sh.#curl -O https://linux.oracle.com/switch/centos2ol.sh#sh centos2ol.sh
To convert Red Hat, however, is quite a bit more steps but fairly easy as well.
The first thing is to remove all of the Red Hat licensing. I shotgunned that with all of these commands:
# subscription-manager list --consumed
# subscription-manager remove --all
# subscription-manager config --rhsm.manage_repos=0
# subscription-manager list --consumed
# subscription-manager remove --serial=[insert serial number]
Next was a bit complicated since the default install did not have wget and was not in the default repository. I used Perl and the Perl module LWP:
#!/usr/bin/perl -w
use LWP::Simple;
getstore('https://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6','/etc/pki/rpm-gpg/RPM-GPG-KEY-oracle');
getstore('http://public-yum.oracle.com/public-yum-ol6.repo','/etc/yum.repos.d/public-yum-ol6.repo');
getstore('http://linux-update.oracle.com/rpms/ULN-CA-CERT','/usr/share/rhn/ULN-CA-CERT');
This worked nicely, not having wget, and I was ready to edit the configuration files. First, note the following changes in /etc/yum.repos.d/public-yum-ol6.repo.
[ol6_u6_base]
name=Oracle Linux $releasever Update 6 installation media copy ($basearch)
baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/6/base/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1 <<==== changed from 0 to 1
Do the same for _latest and _add_ons.
Now edit /etc/sysconfig/rhn/up2date and change /usr/share/rhn/RHNS-CA-CERT to sslCACert=/usr/share/rhn/ULN-CA-CERT. Also, change serverURL setting from https://xmlrpc.rhn.redhat.com/XMLRPC to https://linux-update.oracle.com/XMLRPC.
Now run:
You will see:# yum clean all && yum clean metadata && yum clean dbcache && yum makecache
# yum repolist
# yum list
# yum update
[blah blah]...
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Importing GPG key 0xEC551F03:
Userid: "Oracle OSS group (Open Source Software group)"
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
Is this ok [y/N]: y
...Now reboot and then login and check it out!
Replaced:
redhat-logos.noarch 0:60.0.14-1.el6
Complete!
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.6 (Santiago)
But we really know it's:
# cat /etc/oracle-release
Oracle Linux Server release 6.6
OPTIONAL
I really like EPEL so:# perl -MLWP::Simple -e "getstore('http://mirrors.cat.pdx.edu/epel/6/x86_64/epel-release-6-8.noarch.rpm','epel-release-6-8.noarch.rpm');"
(feel free to use wget if you've got it)
QUESTIONS
Q: Will I have to relabel for SELinux?No, I'm asking you. Not sure, but so far so good so probably not or I'd have noticed an issue by now.
BUGS
[Still trys to look to Red Hat but fails]....
Loaded plugins: product-id, rhnplugin, subscription-manager
There was an error communicating with RHN.
RHN Satellite or RHN Classic support will be disabled.
Error communicating with server. The message was:
Sysid checksum is incorrect
Also, AFTER REBOOT, this annoying nag IF you didn't buy a subscription:
[root@oats ~]# yum update
Loaded plugins: product-id, rhnplugin, subscription-manager
There was an error communicating with ULN.
ULN support will be disabled.
Error communicating with server. The message was:
Sysid checksum is incorrect
Setting up Update Process
No Packages marked for Update
[root@oats ~]# yum update
Loaded plugins: product-id, rhnplugin, subscription-manager
There was an error communicating with ULN.
ULN support will be disabled.
Error communicating with server. The message was:
Sysid checksum is incorrect
Setting up Update Process
No Packages marked for Update
CONCLUSION
As you know, I like it. I get to use Oracle Linux on the small one-offs for free and I have a half dozen systems that will enjoy KSplice - the rest get a basic subscription. Well, that's my opinion (for now).
Monday, March 2, 2015
Book Review: Perl and XML
Recently, I pulled out my (mostly) unread copy of Perl and XML and rather than using it as a reference, began reading the entire thing starting with the Preface. It was a very enjoyable read. The fact that I enjoyed reading it so much was a surprise to me - who would've thought it possible? I had worked with XML and RSS and LDIFs for a dozen years. In the past, I basically just did a quick google and then thought "Yeah, there's the module I need" and installed it an ran with it.
This book explained the differences in the many modules and explained when and where to use them. After nearly eight years I had an "Ah-hah! That's why that program sometimes didn't always catch everything I was looking for!" (I was using a generic XML parsing module when I should've been using XML::DOM.
My only criticism (a minor one) is I would have liked to have seen more and varied examples in the RSS chapter. But, it was an XML book, not a RSS book and the goal was not to show you the difference between Atom and iTunes in RSS feeds, etc.
If you use XML and Perl - you will want to READ this book. Note: I said READ this book. Don't just buy it as a reference and put it on your shelf like I did (at first). In fact, everyone reading this should buy it! (It's only one penny on the used market at Amazon.com. Why not?!)
This book explained the differences in the many modules and explained when and where to use them. After nearly eight years I had an "Ah-hah! That's why that program sometimes didn't always catch everything I was looking for!" (I was using a generic XML parsing module when I should've been using XML::DOM.
My only criticism (a minor one) is I would have liked to have seen more and varied examples in the RSS chapter. But, it was an XML book, not a RSS book and the goal was not to show you the difference between Atom and iTunes in RSS feeds, etc.
If you use XML and Perl - you will want to READ this book. Note: I said READ this book. Don't just buy it as a reference and put it on your shelf like I did (at first). In fact, everyone reading this should buy it! (It's only one penny on the used market at Amazon.com. Why not?!)
Subscribe to:
Posts (Atom)
