LaTeX, Vim and Perl

I'm finishing up grad school.  I write all of my papers in vi (usually Elvis, sometimes Vim) and mark them up with LaTeX.  Yeah, I know.  I'm a geek.  I just love the beautiful look of a typeset document when I'm done.  I like the way it forces me to think about the outline; creating \sections{}, then \subsection{} and even \subsubsection{}.  This looks really nice as an auto-generated table of contents. 

Even better than the beautiful typesetting, I really enjoy keeping my bibliography in BibTeX and having LaTeX/BibTeX automagically write my bibliography citations and print my bibliography at the end by simply writing \cite[p.78]{author}.

And when I'm done, if I don't want a PDF, I can use latex2rtf for an RTF or latex2html to create a web site (or latex2man if I'm feeling extra geeky).  And, I pretty much live in vi all day so, even if I try to use a regular word processor, it gets krufted up with :wq and randomly interjected i's and a's and x', etc.

I used to use emacs.  I did more with emacs I think because I had a Wyse60 monitor and really used the multiple buffers and split screens.  And, loved the macros.  I know, you can do this with vi too.  I really need to learn more of that in vi.  Lately, while writing these papers, I find myself wanting to copy lines from other .tex docs into the current document.  Cutting and pasting outside of vi leaves unexpected broken lines that require a lot of Ctrl-J, x, etc.  Then I saw an article by Jun Wu "Perl as a command line tool"  Lots of fun.  Love Perl one-liners.  She had a version of this example:


    $ perl -ne 'print if $. >=64 && $. <=94' June10.tex >>../FinalPaper/ENG609final.tex 

It's so great to NOT have to clean up line breaks after cut & paste.  And, even better, it gives me an excuse to play around with Perl one-liners.  Some people things that's odd but I see it as recreational - like Soduko or something.

Anyways, that's all I'm up to with Perl lately.  See you all at the 20th anniversary of YAPC. 

Bitten by an old "bug"?

So thankful to find brian d foy's post about LWP::Protocol::https yesterday after some really old scripts just wouldn't work on a new system.  I could tell it was having an issue with the https - but, like Brian (in 2011?!) I thought all of my usual SSL libraries would cover it (which they did).  Installing all of my modules takes about an hour.  So, I continued to flail during this time.  By the time I saw Brian's post, the problem fixed itself (after I installed my own CPAN favorites).

Not knowing my cluster of favorite CPAN modules had fixed this, I quickly did a "install  LWP::Protocol::https" to which cpan replied "installed and already up to date".  "So why isn't it working?!", I thought.  I tried again and it was.

So, to recap, if you plan on using "LWP::Simple qw(getstore)" to grab something from a HTTPS web site and you get a error 501 - this is what you need to fix your problem: LWP::Protocol::https (which was split out in 2011 and I evidently missed the memo).

Perl, Awk or other?

I had some quickie output from an ad hoc Ansible command. I love Ansible! The data was quick to acquire and accurate. But the readability was not optimal. "Hmmm...", I thought, "It'd be good enough if it just separated the data elements with a space." Actually, this comes up a lot with my quickie ad hoc Ansible reports so I do what I always do: Perl!

That worked perfectly but it was so simple I thought it should be a one-liner. BTW, I really enjoyed Peteris Krumins' Sed, Awk and Perl one-liner books. I bought them all in PDF and then bought the pretty No Starch Press version of Perl One-Liners (makes a great coffee table book - I just pick it up and open it to any page and I'm intrigued - Fun!) So, I took a stab at it. It worked perfectly the first time but it was really just my same program in one-liner format. So here it is. If you have a more elegant solution let me know.

  perl -pe 'if (m/SUCCESS/) {print "\n$_"} else {print $_} ' JavaReport.txt

P.S. The I tried Awk:

 awk '/SUCCESS/ { print "\n" } {print}' JavaReport.txt  

(although this isn't quite right)

Being Lazy

Just discovered Array::Utils!  Wonderful!  To think I was going to write a "find unique elements in an array" myself.  Nope!

use Array::Utils qw(:all);

my @unique = unique(@a, @b);

 

 

 

 

These are a few of my favorite modules

I was messing around with Xen on Slackware 14.2 and got my system goofed up.  The console worked but the mouse and keyboard would not function in X.  So, after a bit of flailing, I did a distro upgrade (from Slackware 14.2 to current), re-ran liloconfig and,  ta-dah! -Success!

I noticed Perl didn't work however.  It upgraded Perl but the modules were "mis-matched" it said.  After some googling, I ran across several suggested fixes.  The simplest seemed to me the brute force - delete all Perl references and reinstall Perl - which I did. (perl -E 'map {say $_} @INC')

Now I had a NEW problem.  NO MODULES AT ALL.  Began slowly adding modules as I needed them but today that was getting old.  So I did a quick grep of a few hundred of my Perl programs to see which modules I was using and then did a cpan install of all of them.  It's still running...

Which Perl modules you ask?  Well, to satisfy your curiosity and have this list handy the next time I install a fresh Perl - here's my list:

Gimp Gimp::Fu File::stat Time::localtime DBI Net::LDAP Net::LDAP::Bind Net::LDAP::Search MIME::Lite Net::Nslookup Nmap::Scanner Net::SSL::ExpireDate XML::RSS XML::Simple Test::Simple Net::LDAP::LDIF File::Basename CGI::Carp Net::SNMP SQL::Abstract Text::xSV Pod::HTML2Pod LWP::Simple LWP::UserAgent Net::IMAP::Simple Mojo::UserAgent Data::Dumper Date::Manip Net::Twitter DBD::SQLite Net::DNS Net::Telnet::Cisco DBD::Sybase Crypt::GPG File::Slurp utf8 Socket Net::DNS AnyEvent Net::RabbitFoot WWW::Mechanize HTTP::Cookies File::Copy Bot::BasicBot POE::Component::SSLify Config::General Log::Log4perl::Appender::RRDs LWP::Protocol::https

Wow.  That's actually shorter than I thought.  It looked more impressive as one line.  ;-)

Ansible Playbook and Perl (or 'Salted MD5 - Yummo!')

Not much of an article here - just some notes about encrypting passwords for Ansible playbooks.  All of the examples for adding the encrypted passwords to the ansible playbook are Python (what's up with that?!) :-)

So, I have an example of a playbook to change a user password on all ansible systems and I have translated the password encryption field for Perl (you're welcome!)

Here's my chgUserPW.yml:

---
- hosts: all
  sudo: yes
  tasks:
  - name: Change user1 password
    # Created passwd with:
    # perl -e 'print crypt("your pw here","\$6\$salt\$")'
    # Notes: password, 6 = md5, salt = random string to salt entropy
    user: name=user1 update_password=always password=$6$salt$sjuT2.eSTcX/vKwW7RlB1tdLxyB34lJSsndXA5yzC7BZrdAkiAOqtf4NPtHa0tjdFa/5wcS1.vt0LAwzEassr0

All you do is run the Perl one-liner adding your own password and salt string (the word "salt" is probably not a good choice) and you get the format for the password Linux is expecting (salted MD5).  Paste that into the password field and you can now change all of your system's user1 passwords to "your pw here".

*Disclaimer - I'd try it on a single system first, preferably one you have a backdoor root account on.  It worked for me.  Good luck!

Alternatives to Powerpoint - My Favorite

OpenSource.com just sent me a mailing this morning entitled: 3 open source alternatives to PowerPoint.  I immediately thought to myself "Ok, LibreOffice, CalligraOffice ... I wonder what the 3rd is?".  I was really surprised it was MY personal favorite: Reveal.js.  I've been using that since Perltricks.com had an article on it.  I've seen several presentations done using it at YAPCs and Ohio Linuxfest (coming up soon, btw).

Despite the headline, OpenSource.com goes beyond 3 alternatives and also discusses using your console for slide shows with Vimdeck.  Very interesting!

The article didn't explain how to use (my favorite) Reveal.js though so if you're curious, check out the Perltricks link above.  I think you'll be glad you did.

Alternatives to Powerpoint - My Favorite

OpenSource.com just sent me a mailing this morning entitled: 3 open source alternatives to PowerPoint.  I immediately thought to myself "Ok, LibreOffice, CalligraOffice ... I wonder what the 3rd is?".  I was really surprised it was MY personal favorite: Reveal.js.  I've been using that since Perltricks.com had an article on it.  I've seen several presentations done using it at YAPCs and Ohio Linuxfest (coming up soon, btw). 

Despite the headline, OpenSource.com goes beyond 3 alternatives and also discusses using your console for slide shows with Vimdeck.  Very interesting!

The article didn't explain how to use (my favorite) Reveal.js though so if you're curious, check out the Perltricks link above.  I think you'll be glad you did.

If I HATE web lists so much - why do i click on them?!

I hate web lists.  I really do.  Especially the ones that list things ONE-AT-TIME so you have to load a page that is 90% ads. Irritating!  But, I saw one about "America's Top 100 Colleges". I work at a University - I wondered if we made the top 100.  So, I took the bait.

First I am sent to a page telling me what the list is about WITH LOTS OF ADS and a NEXT button.  I click NEXT. Ok, number 1, lots of ads another NEXT button.  This has already gotten old for me.  I was just curious what the top 100 colleges were - can't they just show me a list?!

So, I think to myself "I wonder if I could just harvest the list from the website with Perl.  I look at the URL - I see what they're doing - simply incrementing the folder by 1 each time.  Easy!  Next I look at the source code: TOO EASY!  They have only the STRONG tag once per page for the school name.  Cool!  So, two minutes of code and then: Tah-dah!


#!/usr/bin/perl -w
# Remember - this took two minutes, I know it's not perfect

use LWP::Simple;

my $rank=100;
my $base="http://www3.forbes.com/forbeswoman/americas-top-100-colleges/";
while ($rank >0) {
    my $url = $base . "/$rank/";
    my $html = get($url);
    $html =~ m{<p><strong>(.*)</strong></p>}
          } or die "Cannot find school: $rank\n";
    $school = $1;
    print "$rank: $school\n";
    $rank--;
}

I run it.  It doesn't work (I think).  It isn't changing the name.  Hmmm.  I keep killing it off before it finishes and examine the code.  I cannot see a problem so let it run - this time all the way to the end.  It does NOT change the name until it counts down to 50.  Wild!  I check the site - sure enough - there are no new schools after 50.  This is a list of the (alleged) Top 50 Colleges going under the name Top 100 Colleges.

Ha!  I guess they never thought anyone would ever have the patience to wade through all of those ads to see all 100 (I sure didn't).  This makes me wonder: Is it just a scam to get you to click through ads or did the content creator tell his boss he put all 100 pages up there knowing NO ONE would ever click through all of that.  If it's the latter - sorry buddy (or lady) for exposing you.

Looking forward to YAPC::NA

I am really looking forward to YAPC::NA.  Looks like it'll be another good one.  I was especially interested in the Plack and Dancer talks (which are not yet confirmed according to the site).  If you see Tatsuhiko Miyagawa or Mark Allen - tell them I'm counting on them! 

Mac OSX and CGI.pm

"I'm NOT a web developer!"  I've been saying that a lot lately.  Even so, because I have done a bit of LDAP programming and they needed a web interface, I wrote a quick hack which turned into a big, long painful project.  The biggest issue is so many things don't work the same (or at all) on Mac OSX.  OpenLDAP, for instance (a whole other story). [It should be noted - I'm also not a Mac guy.]



I tried for several days to get CGI.pm redirects and refreshes to work but kept getting Error 302's.  I found lots of people with the same issue on the Mac.   Finally - someone said "You'll have to write it without CGI.pm because it doesn't 100% work on the Mac (and/or maybe the Apple Apache config was incriminated).  Anyway, I rewrote the redirects without CGI.pm (ugly, inelegant Perl 4ish looking code) and it worked fine.

Since I never exactly found the answer - I want to state for future folks looking for an answer - "IT DOESN'T WORK RIGHT ON A MAC" (this phrase can be re-used for a great many things).  And, if I'm wrong (because, remember, I'm not a web developer) it'd be great for someone to comment with the right answer.

I'll probably get comments such as "You're using CGI.pm for web development?!  Are you from the past?!" To this I would say "Yes I am".

Really good Perl book under a secretive title

Since I've been doing a lot of LDAP programming lately, I finally broke down and bought O'Reilly's LDAP System Administration.  I found a used paperback for $4!  While I was waiting, I got impatient and bought the Kindle version for my Android phone.  I just bought it for the LDAP ACL info but I was thrilled to see chapter 10 was devoted to Perl-LDAP.  It explained a lot of things in the Perl-LDAP example code that I hadn't seen explained before (I'm sure it's out there but the author has done a nice job of digging it all up and putting it together).  Anyway, if you're looking for a nice LDAP book, this is looking like a good one so far (still reading).

New technology I'm Working On

I find that people often think because I work primarily on UNIX/Linux and still use Perl that I'm not learning anything new or that I do not like learning.  Nothing could be further from the truth.  I love new technology and I'm always learning something new.

Here's what I've been working on in the last year:

• Solaris Zones and Containers
• Solaris Zetabyte File system (ZFS)
• GNU Screen
• Ruby
• More Perl LDAP programming
• More Perl programming in general 
• Rakudo (a Perl 6 distro)
• PUGS (another Perl 6 distro)
• SQLite3 and Perl
• RCS (Revision Control System) 

Here's the list of what I hope to cover this year VERY SOON:

• Solaris LDOMs (need a lab experiment)
• See how ZFS is implemented under other OS's (e.g. FreeBSD)
• Use More Ruby! (and I'm looking at a Rails book from the library)
• Juniper's JunOS (I like the XML and the idea of scripting with Perl)
• MORE Rakudo!!  (Not so much Pugs) I hope to contribute bug reports
• Sun's Dtrace (especially using Perl and Dtrace)
• Xen and Linux - need a lab experiment
• Btrfs - Linux's answer to ZFS?  Need to explore in a lab.
• Lots more SQLite (I really like that tool)

 And, of course, my on going quests: (not necessarily in this order)

• Python (Because Zope runs on it!)
• Plone (runs on Zope)
• C - need to refreshen my C programming skills - getting rusty
• Java 
• Command line Sun VirtualBox
• I'm sure there's a few more...

Anyway, this list keeps me pretty busy without doing anything much with any Windows projects.  I can get around fine, IMO, on Windows. Just because I don't have any Windows projects on my plate doesn't mean I'm not learning anything new.  Besides - aren't there plenty of others focusing JUST on Windows?

I'm pretty excited about my list (above) and do as much reading and experimenting  as time permits.  It sure seems to me that I'm learning new stuff - Well, I'm learning things that interest me.  I guess because non-Windows things don't interest some I guess it appears that I'm not learning anything (that interests them).  Oh well.  Can't please everyone and I stopped trying long ago.

Nifty Perl Hack

I threw together a tiny bit of code with the example code from Net::IMAP::Simple to make a program that watches for Mailman unsubscribe messages, logs them, then deletes the message.  It was a 5 minute job (except for the part where my iteration was off by one and kept deleting the wrong message).  Perl is so handy for things like this.

Love the New System Admin w/Perl book (and RCS)!

I've been doing a bit of Perl-LDAP programming and I had always found the 2000 version of Perl for System Administration to be really helpful - especially the tutorials in the appendix (more on that in a bit). I just got the the new version: Automating System Administration with Perl. I was amazed at how much more useful it was. Not only is there 50% more material, ALL of the older stuff has been reworked and freshened. The LDAP section I was using heavily is MUCH improved. Too much goodness to cover now - check it out yourself!

The reason I decided to make the post was to mention RCS, in the books appendix, the author has a 5 Minute RCS Tutorial which makes a strong case for using RCS instead of all of those other revision control systems. I've tried RSC in the past and I've tried CVS and the guys in the office recently tried to get me to use Mercurial (Wow! Talk about overkill for what I do!!) But the authors argument was, this is so simple and easy for your small scripts lying around - why not use it. I was initially daunted by all of the commands and things you could do when I had looked at it years ago but the author distilled it down to a handful of commands. I am going to distill it down further:

1) Check in and unlock your program (leaving a copy):
$ ci -u program
2) To edit, check out and lock your program:
$ co -l program
3) View your revision logs:
$ rlog program

So far, that works well enough for me. Be sure to read the whole 5 Minute RCS Tutorial (see link above). I wish I had used this on my most recent, arduous LDAP programming job. Instead made numbered versions and left them lying all over. I then had to open them to see why this one was named program-4 and how it differed from program-3, etc. I know - silly. Should've stuck with this when I looked at in in 1997 (Hey! I remember the project that caused me to consider rcs then - that's not weird!)

Slackware, Perl, SQLite and SOAP

I was playing around with a project and decided it would be nice to finally try to do something useful with SQLite (and I threw in a SOAP server just-for-fun). I recently reloaded my system with Slackware 13. I was amazed at how easy it was to add Perl modules with Slackware. This is probably making someone out there go "Duh!" but under Fedora and Ubuntu (and Xubuntu, which I use a lot) installing modules on a vanilla system means configuring the CPAN.pm. That's not a big deal but Slackware must have a lot of the CPAN.pm config finished already. It just took off and ran! I had all of the modules I use downloaded in nothing flat and with NONE of the usual haggling I usually have to do to get some of them to load. You have to see it to appreciate it.

Cheap Books!

Wow! This is pretty neat! Too bad I already own both of those. If you didn't catch it - use Perl; says O'Reilly has dropped the price of Learning Perl and Mastering Perl to $9.99 - before other discounts! I have all of the Learning Perl editions! Some are signed by the Author(s). :-p

So what does this mean? I hope it means a 6th edition of Learning Perl is coming out and not something else (remember "cut-out albums"?).

It's tough to say something EVERY WEEK

Man! This is harder than I thought. I don't have any trouble blogging about my racing pigeon training or my catch-all blog (that has revolved around beekeeping lately) . I've got ideas for a couple of other blogs too. But, what to say about my baby-talk Perl that won't be ridiculed? Well, one thing, in response to the co-workers who said "Perl is dead", I thought this news about the new Android phone using Perl spoke for itself!

Am I the only one NOT using "strict"?

An old friend came to town the other day. He's quite a Perl guru. Like me, he was a Perl user prior to Perl 5 and I figured he'd be someone who, like me, NEVER uses 'use strict'; in their code. I was wrong! Man, that's one of the things I like about Perl. It'll let me ride a motorcycle without a helmet. Drive without a seat belt. Go swimming in less than 30 minutes after I've eaten or even run with scissors! I know! I know! These are not good ideas - but short cuts seldom are. It's the price you pay and as long as you know the risks, it's nice that Perl lets you use a variable that has not been declared or invoke $_ with the notion that both Perl and yourself know what you're talking about (in that context). I like that freedom. I guess the other side of the coin is you're also allowed to add "use strict" and "use warnings". I'm just glad it's not the default (yet).

use Net::LDAP::LDIF - no, really, use it!

It took me a while to figure this one out. I guess if I had a better grasp of hashes, this would've been more obvious. The LDIF I was reading was HUGE. I found a good example on PerlMonks that showed this VERY useful line:

$uid = $entry->get_value("uid");

I used DataDumper to find the key names of the other values and then I was off and running: parsing an LDIF filled with 40,000 users and running reports on anything I wanted. On my workstation, it only took 24 seconds to generate multiple reports! VERY COOL!