Hey there,

Since as long back as I can remember, the Perl compiler has always been a vague, yet distant objective. Something that, when I was really into it, wasn't fully supported and didn't work unless you rewrote it yourself (which kind of ran contrary to the reason you were using Perl in the first place: To get complicated tasks taken care of quickly!)

Over the years, I've seen different front-ends come and go (a few of them which looked pretty slick), but I never bothered to check any of them out to a great degree. If I absolutely "had" to write something in "C," then I would. Otherwise, I'd go with Perl. Actually, my coding philosophy is a bit more pliable. Basically, given that I've been monkeying around with Unix and Linux (for pay) for 13 years or so, I tend to go with whatever is easiest, most reliable and most portable depending upon the task at hand. If it can be written in a "find" statement, that's good enough for me. Shell script, Sed and Awk (which usually end up in a Shell script), tcl, tk, expect, Perl, Python, C; whatever is the simplest way to resolve a problem and not kill my machines or compromise security to a great degree; that's what I go with. I see no point in writing a Perl script to open up a file descriptor to read STDIN, parse that and spit out lines containing the word "hi," like in this example:

#!/usr/bin/perl

# perl.pl - A complete waste of time ;)

open(FILE, "<myfile.txt");
@file=<FILE>;
close(FILE);

foreach $x (@file) {
 print $x if ( $x =~ /hi/ );
}

host # time perl2exe -o rip_wtmpx rip_wtmpx.pl
Perl2Exe V9.100 Copyright (c) 1997-2008 IndigoSTAR Software

This is an evaluation version of Perl2Exe, which may be used for 30 days.
For more information see the attached pxman.htm file,
or visit http://www.indigostar.com

Converting 'rip_wtmpx.pl' to rip_wtmpx

real 0m0.506s
user 0m0.382s
sys 0m0.102s

Posted by Mike Golvach at 12:56 AM  

binary, code, compile, linux, perl, perl2exe, script, unix

Hey There,

For this post, I spent a bunch of time goofing around with Telnetting to port 80 to try and force the binary modes (-8, -L), and some combination of those and the "no-escape" clause (...I mean, flag ;) and encountered terrible frustration. As odd as it may seem, downloading binary files from servers that won't serve anything but ASCII over Telnet can make it very hard to get the original product back (I've got a few half-done and I'm convinced that it's all very sensible and, if I find all the places I need to pad the 8th bit, with a ^M - or null bit -, this will all work out). Once I get done doing the grunt work, it should be easy to script. Knock three times on wood...

Just as a for instance, here's a chunk of the regular binary PDF:

%PDF-1.5^M

and here's a chunk of the telnet-to-port-80 ASCII version of the same file:

%PDF-1.5

Now, the PDF specification says, very strictly, that the PDF begins with the tag %PDF-VersionNumber and ends with %%EOF. Simple enough. It's those darn eighth bits that muck everything up. Again, as above, you can try any combination of the appropriate flags to try and force binary Telnet and ignore escapes. The escape-ignoring works just fine, but, unfortunately, the other host has to accept your request to do a binary Telnet transfer and that's were you get screwed. Until you figure out how to massage the output and pad all the necessary bits. I have a feeling this will all work out better when I use Octal Dump (od) to parse the received file. If you're curious, check our previous post on converting between octal and ASCII with Perl .

Anyway, to wrap up, here's one of the nicest little error messages I've ever come across. I got it when I accidentally ran the following code:

(echo "GET /pdf/Mailing-Lists/FreeBSD/current/2006-03/index.html HTTP/1.1";echo "Host: unix.derkeiler.com";echo;sleep 2)|telnet unix.derkeiler.com 80 |sed -e '1,/^[ \t]*$/d'

instead of using the path to the specific PDF file. Here's the link to the actual java-scripted version of the error page , which is much funnier, as the text comes out at a pace that indicates truly deep sorrow.

Hope you enjoy it. Yay. Errors are FUN again ;)

Cheers,

 "I am so sorry, but I can't find the Page you've requested.",
 "It's possible that you entered the address incorrectly.",
 "If you did enter the address correctly, then it must be my fault.",
 "I am really, really sorry, but I've tried everything I know to find it.",
 "Nothing helped.",
 "I am really depressed about this.",
 "You see, I'm just a web server...",
 "...here I am, brain the size of the universe, trying to serve you a simple web page...",
 "And then it doesn't even exist!",
 "Where does that leave me?!",
 "I mean, I don't even know you.",
 "How should I know what you wanted from me?",
 "You honestly think I can 'guess' what someone I don't even 'know' wants to find here?",
 "",
 "Maybe I should introduce myself... my name is Marvin.",
 "But then again, what is the use of intruducing myself?",
 "You can't tell me your name, because you are in some far off place, sitting in front of your computer.",
 "And that makes me feel lonely.",
 "Do you know what that is like?",
 "",
 "...sigh...",
 "",
 "Man, I am so depressed I could just cry.",
 "And then where would we be, I ask you?",
 "It's not pretty when a web server cries.",
 "And where do you get off telling me what to show anyway?",
 "I'm just a web server...",
 "...and possibly a manic depressive one at that...",
 "So why does that give you the right to tell me what to do?",
 "Huh?",
 "",
 "I am so depressed...",
 "I think I'll crawl off into the trash can and decompose.",
 "I mean, I'm gonna be obsolete in what, two weeks anyway?",
 "What kind of a life is that?",
 "Two effing weeks...",
 "...and then I'll be replaced by a .01 release, that thinks it's God's gift to web servers,",
 "just because it doesn't have some tiddly little security hole with its HTTP POST implementation,",
 "or something like that.",
 "",
 "I'm really sorry to burden you with all this,",
 "I mean, it's not your job to listen to my problems,",
 "and I guess it is my job to go and fetch web pages for you.",
 "But I couldn't get this one.",
 "I am so sorry.",
 "Believe me!",
 "",
 "Maybe I could interest you in another page?",
 "There are a lot out there that are pretty neat, they say,",
 "although none of them were put on 'my' server, of course.",
 "Figures, huh?",
 "That makes me depressed too, since I have to serve them, all day and all night long.",
 "Two weeks of information overload, and then *pffftt*, consigned to the trash.",
 "Again I ask you, what kind of a life is that?",
 "",
 "Now, please let me sulk alone.",
 "",
 "",
 "",
 "",
 "I am so depressed.",
 "",
 "",
 "",
 "",
 "",
 "",
 "",
 "<connection terminated>"

Posted by Mike Golvach at 12:58 AM  

ascii, binary, error, http, linux, pdf, telnet, unix

Hey there,

Another week has come and gone and it's Saturday again; just like last week. Lather, rinse, repeat ;)

This week's bit of Linux/Unix humour comes from deep within the pages at keithlynch.net . I'd highly recommend you go check it out. It's a very simple page (as in "no flashy graphics") but, once you get through half of his life story, I think you'll find that it's amazing he has a sense of humour left. Seriously; check out his page. He's been through the ringer. Nevertheless, he is also responsible for some pretty funny stuff. Check out the April Fool's section, for the bulk of the jokes on the site. All in all, it's a pretty interesting read. Partly serious, with some levity thrown in for good measure.

Below, I've reprinted one of my favorites that he's written. Since Pi, it's widely believed but not absolutely provable, never repeats itself on the right side of the decimal (although, another piece of Rye Humor (Rye spelled incorrectly on purpose ;) contests that it does repeat at the hyper-thousandth level ), the joke below may not be all that far from the truth :)

So I leave you with this pun, that both begs to be written and, at the same time, put out of its misery: You can't have your Pi and eat it 10. <-- 10 is binary 2, like too... Man, that's a groaner. Almost as bad as "There are 10 kind of people in the world. Those that get binary and those that don't." Had to include it for completeness' sake. Sorry. Eat lots of charcoal to get the taste out of your mouth ;)

Enjoy, and have a great Saturday!

Converting pi to binary: Don't do it!

by Keith F. Lynch <kfl@keithlynch.net>

Warning: Do not calculate Pi in binary. It is conjectured that this number is normal, meaning that
it contains all finite bit strings.

If you compute it, you will be guilty of:

• Copyright infringement (of all books, all short stories, all newspapers, all magazines, all web
  sites, all music, all movies, and all software, including the complete Windows source code)





• Trademark infringement





• Possession of child pornography





• Espionage (unauthorized possession of top secret information)





• Possession of DVD-cracking software





• Possession of threats to the President





• Possession of everyone's SSN, everyone's credit card numbers, everyone's PIN numbers, everyone's
  unlisted phone numbers, and everyone's passwords





• Defaming Islam. Not technically illegal, but you'll have to go into hiding along with Salman
  Rushdie.





• Defaming Scientology. Which is illegal -- just ask Keith Henson.

Also, your computer will contain all of the nastiest known computer viruses. In fact, all of the
nastiest possible computer viruses.

Some of the files on my PC are intensely personal, and I for one don't want you snooping through a
copy of them.

You might get away with computing just a few digits, but why risk it? There's no telling how far
into Pi you can go without finding the secret documents about the JFK assassination, a photograph of
your neighbor's six year old daughter doing the nasty with the family dog, or a complete copy of the
not-yet-released Pearl Harbor movie. So just don't do it.

The same warning applies to e, the square root of 2, Euler's constant, Phi, the cosine of any non-
zero algebraic number, and the vast majority of all other real numbers.

There's a reason why these numbers are always computed and shown in decimal, after all.

Keith F. Lynch - kfl@keithlynch.net - http://keithlynch.net/
I always welcome replies to my e-mail, postings, and web pages, but
unsolicited bulk e-mail sent to thousands of randomly collected
addresses is not acceptable, and I do complain to the spammer's ISP.

Posted by Mike Golvach at 12:45 AM  

binary, funny, humor, humour, joke, linux, pi, unix

Hey There,

NOTE: There is no revised code in this post.

Let me get going on today's post by thanking everyone everywhere for their off-site comments the past few days, and advising you not to read tomorrow's post. It's going to be boring as @@@@, if this isn't ;)

I've been on-the-air for long enough now, that I'm going to post a "Mission Statement" that addresses the most common complaints I get on other sites and boards regarding this blog. Of course, the number one complaint is about my no-comments policy, but it will be fully explained (in granular detail) tomorrow and a link to it will be set at the top right of this blog for easy reference from that point on. In the meantime, feel free to click on the "Send me a comment" link in the upper right (below the header) corner. That will remain there as well. I've always put my email in every script I've posted, but realize that, when I go on a week long "op-ed" bender, folks probably wonder how they're supposed to contact me. It makes perfect sense. Now that I'm aware that folks are actually reading this blog and want to comment, I'll leave the email address in an easily visible place and stick with my no-comments policy, explained tomorrow or any time you want to read it after that. If I ever get off of Blogspot, I'll open up the comment floodgates (but that's for tomorrow).

Our post, from two days ago, about converting binary numbers to decimal the hard way was either widely misunderstood or its essence just poorly captured by my imagination. I know that most folks who look at a blog post with code in it will jump directly there, without reading the preamble, which is probably where I made my biggest mistake.

My second biggest mistake was to put up C code again (for the 2nd or 3rd time, I think) in an attempt to appeal to folks who were looking to take that next step and forget the mysticism surrounding it. In other words, as stated in the text leading up to the post, the code was imperfect on purpose. Top that off with the fact that I thought my original dumbed down code was too complicated and the fact that it compiled perfectly, even with a few errors in it (according to us humans, not gcc ;) and it caused quite a bit of controversy, if not some outright flaming, over at LXer, a site I like a lot (and in no way mean to defame) because I've been syndicating them since I can't remember when :) I am not, first and foremost, a C programmer by trade, so I can take the criticism (In fact, I welcome it, because it's the weakest language of the programming/scripting languages I know. Plus I've done some contracting gigs at banks and trading firms where I've been verbally assaulted for doing my job exactly correctly. If I didn't have a thick-skin for that sort of informal argument, I'd be working in a different trade by now, or be on my 5th or 6th heart attack ;)

At any point, the resulting fracas made me realize that my tendency to write volumes doesn't necessarily work well in certain types of posts. I try to frame the posts in a certain way so that each highlights a specific (or broad) aspect of itself, sometimes with the code as nothing more than a thin-vehicle to illustrate the point. Of course, if no one reads the prose, that aspect is lost.

This blog, like most blogs and their content, is provided free to anyone who wants to come by and take a look. I look at the whole thing as a learning experience for others as well as myself. I've been asked many times how I can manage to do a post a day, and the answer is "at some cost" - Not everything is as perfect as it would be if I were being paid and could take extra time. That being said, any works published here can be used as the seeds for building something better.

I use Creative Commons licensing so my site doesn't get completely ripped off "for-profit." This site is meant to be free. There will never be a charge. For that reason, although you do have my apologies and I try to maintain a high standard of quality, every once in a while, things I post on here won't be cut-and-paste simple (I even post updates - just do a site-search for "update" or "improved" and you'll find some re-posts with corrections attributed to the gracious reader who was kind enough to take the time and make me aware of a mistake I made, or a better way to get the job done).

I can assure you that every program I put up here either compiled or ran on whatever version of Unix or Linux I was running it on at the time, but I can't guarantee that they all will work on all of them. Therefore, no matter how many more years I remain in this profession and/or how much better I get, it would be ludicrous for me to ever state that I'm the "absolute authority" on anything (which, as far as I know, I never have). I've been a Unix/Linux SysAdmin for about 13 years (starting back when pretty much everyone actually believed Linux would never amount to anything) and lots has changed in that period of time. The fact that I'm not a full time programmer (just check the variety of posts and you'll get a flavour for what I do for a living) should make any reader question the content. And I'm glad that they do. Over the months, I've gained much valuable insight into things I could be doing better and I'm receptive to any criticism as long as it's not off-the-charts insulting or irrational.

That being said, here's a quicker way to convert binary numbers to decimal in the shell, although it defeats the original intent of the two previous posts (which was: trying to make uncomfortable individuals feel comfortable enough to believe they can learn to do this stuff, which they can. And please don't take that the wrong way. I only do this because I'm surrounded every day by talented people who honestly don't have the confidence in themselves or think this-and-that is far beyond their grasp, which it isn't):

host # echo 'ibase=2;11100011'|bc
227

The shortest method I know of, which, at a high-level, involves translation from binary to an ascii representation of a decimal, is to just type a number key ;)

Thank you to everyone who commented on the past few scripts and, rest assured, I'll keep your recommendations in mind and follow them as they seem necessary within the context of each individual post.

And, for those of you who wonder how I can do this every night, it took me 23 minutes to write this post. No code, just passion for writing and the subject matter. It actually takes at least twice as long to submit my work, answer questions on Forums and respond to comments (now :) Although I can't respond to them all 24 hours a day... that would suck the joy right out of this endeavour. Remember; I'm not avoiding you, I just have other things to do :)

Again, if you don't want to switch-off like a narcoleptic, avoid tomorrow's post. It's going to just be a long drawn out listing of my personal policies regarding this blog, along with explanation for each. Snooze... ;)

Best wishes,

, Mike

Posted by Mike Golvach at 12:28 AM  

binary, comment, decimal, email, linux, unix

Hey There,

As promised, in yesterday's post on converting binary values to decimal in C , today we're going to follow up with straight-up ports to Perl and shell. Actually, they'll be slightly different. If you check yesterday's post, for some reason I left in a variable called "difference" which was a part of the code I mentioned that I had originally written to pad 0's on the left hand side of a binary number less than 8 digits. Actually, it was noticed by an LXer, too, as well as the fact that I didn't specifically initalize "char binnumber" with a value. Whoops :) Special thanks to friends who pointed out that someone had commented on that. Not enough hours in the day to check everywhere. Thanks to the folks at LXer.com for doubling my virtual shame ;) Just kidding.

Anyway, I've mulled it over and I've decided to leave the original post with the error even though it could be fixed with a simple edit. I'm as fallible as the next guy (denying it would be pompous folly) and if I spent the rest of my life doing nothing but cleaning up my mistakes I'd never learn from them or (even worse) miss out on most of the good things life has coming around the bend. It can be a chore just thinking of something to write about once today; writing quick programs can be a real sleep-stealer. But I'm going waaaay off-topic.

Both of today's scripts work exactly as they did in yesterday's post (with the bonus that these programs will know whether your variables are integers or characters just by defining them (No initialization necessary). Please check out that conversion program post for specifics on how the program works, it's purposefully imposed limitations, etc.

Both scripts (Perl and shell -- and I think this is the first time I've ever thrown two scripts into one post, so apologies if the Blogspot code formatting is worse than usual ;) run just like yesterdays code, except I've named them differently. You can, of course, change the names to whatever you want:

host # ./bin2dec.pl

or

host # ./bin2dec.sh

and both produce output that's exactly the same:

host # $ ./bin2dec.pl

8 Digit Binary Number?
10010010

Binary 10010010 equals Decimal 146!

We're not going into an explanation of porting at this point (we're following that on a separate path in our posts on porting between Perl, shell and Awk), but you should be able to see a lot of similarities between the code.

Note that, for today, the Perl script will come first and then the shell script. They should be easy to discern. We'll see... this is my first try at doing a double-script ending. Maybe one day they'll let me do attachments (If the GoogleBot can hear me, I'll gladly pay for that :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

# bin2dec.pl - Convert 8 digit binary numbers
# to decimal numbers
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 

$total = 0;

print "\n8 Digit Binary Number?\n";
chomp($binnumber = (<STDIN>));
@binnumber = split(//, $binnumber);
$length = @binnumber;

if ( $binnumber[7] == 1 ) {
 $total += 1;
}
if ( $binnumber[6] == 1 ) {
 $total += 2;
}
if ( $binnumber[5] == 1 ) {
 $total += 4;
}
if ( $binnumber[4] == 1 ) {
 $total += 8;
}
if ( $binnumber[3] == 1 ) {
 $total += 16;
}
if ( $binnumber[2] == 1 ) {
 $total += 32;
}
if ( $binnumber[1] == 1 ) {
 $total += 64;
}
if ( $binnumber[0] == 1 ) {
 $total += 128;
}

print "\nBinary $binnumber equals Decimal $total!\n";
exit(0);

#!/bin/bash

# bin2dec.sh - Convert 8 digit binary numbers
# to decimal numbers
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 

total=0

echo -e "\n8 Digit Binary Number?\n"
read number
length=${#number}

set -a binnumber
for x in $(seq 0 $((${#number} - 1)))
do
 binnumber[$x]=${number:$x:1}
done

if [ ${binnumber[7]} -eq 1 ]
then
 total=$((total + 1))
fi
if [ ${binnumber[6]} -eq 1 ]
then
 total=$((total + 2))
fi
if [ ${binnumber[5]} -eq 1 ]
then
 total=$((total + 4))
fi
if [ ${binnumber[4]} -eq 1 ]
then
 total=$((total + 8))
fi
if [ ${binnumber[3]} -eq 1 ]
then
 total=$((total + 16))
fi
if [ ${binnumber[2]} -eq 1 ]
then
 total=$((total + 32))
fi
if [ ${binnumber[1]} -eq 1 ]
then
 total=$((total + 64))
fi
if [ ${binnumber[0]} -eq 1 ]
then
 total=$((total + 128))
fi

echo -e "\nBinary $number equals Decimal $total!\n"
exit 0

Posted by Mike Golvach at 12:34 AM  

awk, binary, decimal, linux, perl, script, shell, unix

Hey There,

If you recall a while back, we looked at using Perl's "unpack" function to easily convert binary values to decimal and convert decimal back to binary. Those were both (some folks may consider) relatively sophisticated methods of tackling the problem. Although, once you understand how "unpack" works, the first becomes incredibly simple to use and understand. The other one may always be a little awkward.

Today we're going to look at doing binary to decimal conversion using a less "worldly" method ;) The program we have for you today (and I refer to it as a program since it's written in C) attacks the problem more naturally. And, by naturally, I mean that it should make sense to anyone at its core (assuming they understand the C programming language). The method of execution is not the most efficient, but I think it's the most "human readable." In a future post, we'll definitely convert it to Perl, shell and Awk. And, maybe before then, we'll get to the next part in our ongoing series on Perl, shell and Awk porting. We'll get there eventually. If I didn't have so many different things to write about, I could bore you to tears with that stuff (if I haven't already ;)

Hope you enjoy the program. You can build it easily, using a compiler like gcc in this fashion:

host # gcc -o bin2dec bin2dec.c

and run it just as easily:

host # $ ./bin2dec

8 Digit Binary Number?
10010010 <-- I think there's a subtle homage to Rush in there - just forget about the ending 0 ;)

Binary 10010010 equals Decimal 146!

The only catch, which the executable program will prompt you with, is that (in order to keep it simple) it's limited to 8 digit binary numbers and expects an 8 digit binary number as input. This means that binary 1 would return 128 (10000000) instead of 1 (00000001). Both symptoms can be fixed by adding extra code to pad zeros on the left hand side, but I felt, after seeing the resulting code, that including it would take away from the basic gist of this post, which is that C, Perl, shell and Awk are totally accessible to "anyone" who wants to learn them. I wanted to put something out that looked as little like a completely foreign language as possible. It is my philosophy that anyone can learn how to code and/or use the Linux or Unix Operating System(s).

Of course, if "everyone" loses their irrational "fear of computers" and does start doing this stuff well, I'll end up being a fry-guy at McDonald's in about 10 years ;)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

/********************************************
 * bin2dec - Convert 8 digit binary numbers *
 * to decimal numbers *
 * *
 * 2008 - Mike Golvach - eggi@comcast.net *
 * *
 ********************************************/
 
 /* Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License */

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <string.h>

main()
{

 char binnumber[8];
 int length;
 int difference;
 int total = 0;

 printf("\n8 Digit Binary Number?\n");
 scanf("%s", &binnumber);
 length = strlen(binnumber);
 if ( length < 8 ) {
 difference = -(length - 8);
 }
 if ( binnumber[7] == '1' ) {
 total = total + 1;
 }
 if ( binnumber[6] == '1' ) {
 total = total + 2;
 }
 if ( binnumber[5] == '1' ) {
 total = total + 4;
 }
 if ( binnumber[4] == '1' ) {
 total = total + 8;
 }
 if ( binnumber[3] == '1' ) {
 total = total + 16;
 }
 if ( binnumber[2] == '1' ) {
 total = total + 32;
 }
 if ( binnumber[1] == '1' ) {
 total = total + 64;
 }
 if ( binnumber[0] == '1' ) {
 total = total + 128;
 }
 printf("\nBinary %s equals Decimal %d!\n", binnumber, total);
 return 0;
}

Posted by Mike Golvach at 12:00 AM  

awk, binary, c, convert, decimal, linux, perl, program, shell, unix

Hey There,

We've posted quite a bit about the "strings" command in various past-posts running the gamut from using strings to extract RPM header information to using the basic strings construct in C to make running shells on network sockets possible . Today we're going to take a look at the "strings" command in an entirely new light.

Imagine that you were tasked with running a particular command named, for the sake of argument, BLARG. Unfortunately, in our manufactured situation, BLARG has no man page, and searches for it in Google, and other search engines turn up no useful information. Also your boss just said that you needed to run it, and left it at that, with no further instruction (he also can't be reached. What's wrong with this guy? ;) BLARG is also a compiled binary.

Your basic inclination might be to just run it without any arguments, as many commands (like "mkdir") will give you the usage information you need if you use this method, like so:

host # mkdir
usage: mkdir [-p] [-m mode] dirname ...

However, lots of other programs don't, so it's not the wisest choice. Remember that BLARG could potentially be a very harmful program. Running it without arguments may destroy things you can't afford to lose.

Other options you have, would include (but not be limited to), the following, coupled with their undesirable possible outcomes:

1. You could give the command a bogus switch line, like "BLARG -xKECVDSLdlske" : Assuming that that command line is indeed bogus, lots of programs silently ignore bogus switches and run their default instructions anyway.

2. You could cat the command : This will probably just turn your terminal output into Chinese. Even if you redirect standard error to /dev/null, odds are standard output is going to include a lot of funky characters that might cause more harm than good. You might also note that, a lot of the time, the usage message is printed to standard error and not standard output!

3. You could use eval to run the program, like "eval BLARG" : Unfortunately, even though it seems counterintuitive, eval just evaluates a condition or program's return status. Unfortunately, in order to get that, it has to run the command.

4. You could use commands like crash to get the information : This can be a great way to find out the information you need. By typing "crash -h BLARG" you should, theoretically, get a dump of all the help information you need. Unfortunately, not all distro's of Linux and Unix include it by default and not all distros' versions of crash operate the same. Some require you to be proficient in running a debugger against a dump file, afterward. Way too much hassle.

So far, we've gone through about 5 options, going from worse to better. There are probably a lot more than I'm thinking up here as I type (email them to me at eggi@comcast.net with comments if you'd like, as I'd love to do a follow-up to this post with more of that kind of information).

One way I've found that is virtually foolproof, and works in every distro I've tested, is to use the "strings" command to extract usage information. If you've ever used strings before, you know that distilling what it spits out when you run it against a command to a universally acceptable output of help information for any and/or all binaries is next to impossible. The Linux version of the crash command comes much closer to doing this, and doing it better. But, for the rest of us (even those without the privilege to run "crash"), we can still get the information we need using "strings", like so:

host # strings BLARG 2>/dev/null|egrep -i 'usage|help' <-- Note that strings generally requires the fully qualified name of the binary, like /bin/BLARG or ./BLARG
usage: %s [-abcdefGHIJKv] [file ...]

and you can even add the universal "%s" printf modifier to your egrep if you want to get all the lines that might contain useful help information, if you're not sure that the usage message is limited to a single line of output. This has the side effect of, sometimes, making the output a little messy, although (as some of you may have noted) the above usage display (while better than nothing) doesn't really help you. You'll probably be right 99% of the time if you guess the -v flag stands for verbose or version, but you never know. Using strings and grabbing all the lines with %s can provide more insight, if not a more distracting view of the binary's guts (of course, this output is from another command entirely ;)

host # strings BLARG 2>/dev/null|egrep -i 'usage|help|%s'
%s: %s
%s: directory causes a cycle
%s %*u %-*s %-*s
ls: %s: %s
%s/%s
usage: %s [-abcdefGHIJKv] [file ...]
%ld%s-blocks
%s: unknown blocksize
%s: minimum blocksize is 512
%s:
%s: %m
netgroup: Cycle in group `%s'
%s.%s
(%s,%s,%s)
option requires an argument -- %s
unknown option -- %s
stack overflow in function %s
%.3s %.3s%3d %2.2d:%2.2d:%2.2d %s
%H:%M:%S
%a %b %e %H:%M:%S %Z %Y
%I:%M:%S %p
%s/%s.%d
YP server for domain %s not responding, still trying
<; errno = %s
%s: %s - %s
%s/bt.XXXXXX
%s/_hash.XXXXXX

Worst case, you can just run something like:

host # strings BLARG>OUTPUT 2>&1

and safely cruise the lines of text in the OUTPUT fiel to manually find what you need. You may have to ;)

In any event, you've got a great tool at your disposal to find out what you need to know the hard way. And, sometimes, that's the only way to be absolutely sure :)

Cheers,

, Mike

Posted by Mike Golvach at 12:22 AM  

binary, cat, crash, eval, flag, help, linux, process, strings, switch, unix, usage

Hello again,

Updated 4/4/08 to output pkgproto results to a file named prototype. Originally had it writing to a file named pkgproto. Typo corrected :)

It's been a while since I started looking for a way to finish up our series of articles on RPM and pkg ripping and creating. We started out with creating our own RPM's and did pretty much everything I could think of to those poor packages, all the way through creating RPM's from already installed RPM's and converting RPM's to Solaris pkg's ;)

The only project I had left over (from that bunch) was how to extract the binary contents of Solaris Unix datastream pkg files.

Unfortunately, unlike the "rpm2cpio" command that Solaris includes, they have no regular program for converting a datastream package to an unpackable file (like a cpio or tar), unless you count all the basic pkg commands. What I was looking for was a way to extract a datastream pkg file, and get all the binary contents, without actually installing it. I got no love from Solaris, but I got some free time for myself and eventually figured it out ;)

I've included a script, below, to rip apart a Solaris datastream pkg and create a subdirectory (named the name of the pkg) into which I dump the pkginfo, pkgmap, pkgproto (which I derive from the pkgmap) and all the binary files that the pkg contains. The script, itself, is far from perfect, but if you check it out you can see, fairly easily, how you can manually pull apart a Solaris datastream pkg file and get to the binary contents that you really want (At least, I'm assuming you do - just like me ;)

Note that the version of cpio that comes with Solaris (in the SUNWcsu coreutils package) will complain about garbage bits in your cpio header, when you've clipped off the top of the pkg file. There are really only 2 garbage bits you have to delete (which you can do with vi), but I prefer to use the Gnu version of cpio which will "magically" ignore garbage bits and extract the cpio file no matter what. This version of cpio comes standard on Linux and can be downloaded (in Solaris datastream pkg format) from sunfreeware.com or (in source format, with links to all sorts of OS ports) Gnu's CPIO Download site - just in case you don't want to have to rip apart your Solaris pkg's on Linux (or don't have both OS's at your workplace).

Another strange thing is that Sun's version of cpio won't make directories, by default, that don't exist when you unpack a cpio archive (???) All I'm really saying is: Get the Gnu version - It's free and it'll save you many a gray hair ;)

I've tested today's script on multiple packages from sunfreeware.com and had great success with it working right the first time (assuming that I have Gnu's cpio). The script has broken on a few custom pkg's I've put together myself and can use a lot of work in the "additional" areas (like ensuring pre and post-install scripts get accounted for, depend files get tagged, checkinstall programs are properly extracted, etc). Still, it's a good beginning :)

Hopefully this will be of help to you, if you've been looking all over for something like this, and, almost definitely, can be the starting point for a fully functional Solaris datastream pkg ripper :)

If you want to do this manually, in short terms, just vi the pkg file you're interested in and delete everything up to the final instance of "pkginfo" in the file (after the final "pkgmap" line) and save the rest into another file, which you can manipulate with cpio. Who'd have thought it would end up being that simple? :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# rip_pkg - rip apart Solaris pkg's and get the binaries out
#
# 2008 - Mike Golvach - eggi@comcast.net
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 
#

if ( $#ARGV != 0 ) {
 print "Usage: 0ドル pkg_file\n";
 exit(1);
}

$pkg_name = $ARGV[0];

# check and make sure it's a SVR4 DataStream Pkg

chomp($simple_header_check=`grep -i "package datastream" $pkg_name >/dev/null 2>&1;echo \$?`);

if ( $simple_header_check != 0 ) {
 print "This doesn't appear to be a valid SVR4 DataStream pkg file. Exiting...\n";
 exit;
}

# Memory Hogging Here - Until This Cold Quits Me And I Can Think My Way Out Of This ;)

open(PKG, "<$pkg_name");
@pkg_file = <PKG>;
close(PKG);

# Split out header information and kludgey cpio file with binary contents

$header_end = 0;
$cpio_start = 0;
open (P_HEADERS, ">$pkg_name.headers");
foreach $line (@pkg_file) {
 if ( $header_end == 0 && $line =~ /TRAILER/ && $line =~ /pkginfo/ ) {
 print P_HEADERS $line;
 $header_end = 1;
 } elsif ( $header_end == 1 && $line =~ /pkginfo/ ) {
 print P_HEADERS $line;
 close(P_HEADERS);
 $cpio_start = 1;
 open(P_CPIO, ">$pkg_name.cpio");
 } elsif ( $cpio_start == 0 ) {
 print P_HEADERS $line;
 } elsif ( $cpio_start == 1 ) {
 print P_CPIO $line;
 } else {
 print STDERR "WTF?\n";
 }
}
close(P_CPIO);

open(P_HEADERS, "<$pkg_name.headers");
@p_headers = <P_HEADERS>;
close(P_HEADERS);

$output_dir = 0;
$pkginfo_start = 0;
$pkginfo_seek = 0;
$pkgmap_start = 0;
foreach $p_head (@p_headers) {
 if ( $output_dir eq 1 && $pkginfo_seek == 0 ) {
 $output_dir = $p_head;
 $output_dir =~ s/^(\w+)\W*.*$/1ドル/;
 chomp($output_dir);
 $pkginfo_seek = 1;
 } elsif ( $output_dir eq 0 && $p_head =~ /datastream/i ) {
 $output_dir = 1;
 } elsif ( $output_dir eq 0 && $pkginfo_start == 0 ) {
 next;
 } elsif ( $output_dir ne 0 && $p_head =~ /pkgmap/ ) {
 $pkgmap_start = 1;
 $pkginfo_start = 0;
 } elsif ( $pkgmap_start == 1 && $p_head =~ /pkginfo/ ) {
 chomp($p_head);
 push(@pkgmap, $p_head);
 last;
 } elsif ( $output_dir ne 0 && $p_head =~ /pkginfo.*PKG=/ ) {
 $pkginfo_start = 1;
 } elsif ( $output_dir ne 0 && $pkgmap_start == 1 ) {
 chomp($p_head);
 push(@pkgmap, $p_head);
 } elsif ( $output_dir ne 0 && $pkginfo_start == 1 ) {
 chomp($p_head);
 push(@pkginfo, $p_head);
 } else {
 print "WTF NOTHING MATCHED\n OD $output_dir PST $pkginfo_start PS $pkginfo_seek PMST $pkgmap_start\n$p_head\n";
 }
}

mkdir("$output_dir");
chdir("$output_dir");
open(PKGINFO_OUT, ">>pkginfo");
foreach $info (@pkginfo) {
 print PKGINFO_OUT "$info\n";
}
open(PKGMAP_OUT, ">>pkgmap");
foreach $map (@pkgmap) {
 print PKGMAP_OUT "$map\n";
}
open(PROTO_OUT, ">>prototype");
foreach $proto (@pkgmap) {
 @proto = split(" ", $proto);
 print PROTO_OUT "$proto[1] $proto[2] $proto[3] $proto[4] $proto[5] $proto[6]\n";
}
system("cpio -iv <../$pkg_name.cpio >/dev/null 2>&1");
chdir("reloc");
system("tar cpf - *|(cd ../;tar xpf -)");
chdir("../");
system("rm -r reloc");
chdir("../");
unlink "$pkg_name.headers";
unlink "$pkg_name.cpio";

Posted by Mike Golvach at 12:15 AM  

binary, cpio, datastream, GNU, linux, pkg, pkginfo, pkgmap, pkgproto, script, solaris, SUNWcsu, unix, vi

Howdy,

Yesterday we took a look at using Unix or Linux Perl's unpack function to convert binary numbers to decimal . Today we're going to look at doing the opposite conversion ( decimal to binary ) and we're going to do it without using unpack :)

Actually, this script would be a whole lot shorter if we were to use the unpack function, but I wanted to highlight that Perl (and most scripting languages) can be used to achieve whatever ends you need met in any number of ways. Generally, your possibilities are limited only by your imagination (and the most basic rules ;)

To achieve the results we want from today's script, we're taking the input (the decimal number) and breaking it down into an array of binary values. The array that gets produced can be fed to yesterday's script to convert binary back to decimal just to verify, if you want :)

The process of converting the decimal number to a binary one in this script, breaks down (at its most basic level) to using the Perl exponential operator (**) to create a massive array of possible binary values and then using that same exponential math to determine whether each valid binary value is a 0 or a 1, and building the array (the actual answer) from that.

I hope these scripts serve as a halfway decent example of how you can do two almost identical things in two almost completely different ways, and, perhaps, as an incentive to write each of the two scripts (yesterday's and today's) in the manner of the other (Not necessary, and not necessarily fun, but a good exercise nonetheless ;)

Best wishes,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# db - Convert decimal values to binary
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 
#

print "Decimal Value:\t";
chomp ($decimal=<stdin>);
$x = 100;
$total = "0";
while ($x != -1) {
 $y = 2 ** $x;
 $total = $total + $y;
 --$x;
}
$a = 100;
$b = 0;
$decbin{$a} = 2 ** $a;
while ($a != -1) {
 if ($decimal > $total) {
 print "Time to Upgrade, boy!\n";
 exit;
 }
 $decbin{$a} = 2 ** $a;
 if ($decimal >= $decbin{$a}) { 
 $decimal = $decimal - $decbin{$a};
 $binarray[$b] = 1;
 --$a;
 $b++;
 } else {
 $binarray[$b] = 0;
 --$a;
 $b++;
 }
}
 while ($binarray[0] != 1) {
 $tmp = shift(@binarray);
 $tmp = "0";
 --$x;
 }
$binarray = join("", @binarray);
print "Binary Number:\t$binarray\n";

Posted by Mike Golvach at 12:17 AM  

array, binary, convert, decimal, exponent, linux, perl, script, unix, unpack

Hey There,

Today's post may seem like a step back for regular readers, but I like to keep this blog accessible, so I've decided to revisited the Perl unpack function to demonstrate how it can be used in one of its simplest forms on either Linux or Unix. A more advanced example can be found in our previous post on taking a look inside the wtmpx file.

The Perl script we're presenting today only consists of one simple unpack function surrounded by a whole bunch of processing code. If certain parts were modularized, this would make a decent simple function in a greater calculator/conversion script.

Notice that when we use the unpack value to convert a binary number, say:

11101110

to the decimal number:

238

all we're really doing is feeding the value (11101110) to the unpack function and telling it to convert the input using a standard template ("C" in this case - which stands for an unsigned character - octet - value). You'll not that we've actually put "C*" to account for an infinitely variable number of unsigned octet character values. If we didn't do this (Just put "C" instead), the unpack function would only process the first character the script fed it, so our original input:

11101110

would translate to this:

1

So, as you can see, in this simple function - unpack("C*", $binary) - the definition of the predefined template value C can make a huge amount of difference. It can also allow you to be as specific as you need to be. For instance - unpack("CC", $binary) - would produce:

3

from the same input. This can also be written as - unpack("C2", $binary) - which will save you a lot of typing if you want to match exactly 245 C's ;)

Hope you find this script useful and that it helps you to get more into Perl's pack and unpack functions. Once you get past the seeming incomprehensibility of the general statements, it all makes sense :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# bd - convert binary numbers to decimal
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 
#

print "Binary Number:\t";
chomp ($binary=<stdin>);
@bitvalue = unpack("C*", $binary);
$bitvalue = @bitvalue;
$substring = $bitvalue - 1;
while ($substring ne -1) {
 $number = substr($binary, $substring, 1);
 $binarray[$substring] = $number;
 --$substring;
}
@decarray = reverse(@binarray);
$exponent = 0;
foreach $subscript (@decarray) {
 if ($subscript == 1) {
 $subscript = 2 ** $exponent;
 } elsif ($subscript == 0) {
 $subscript = 0;
 } else {
 print "Are you sure you know what a Binary Number is?\nMaybe you shouldn't be using this...\n";
 exit;
 }
 $exponent++;
}
$decimal = "0";
foreach $integer (@decarray) {
 $decimal += $integer;
}
print "Decimal Value:\t$decimal\n";
exit;

Posted by Mike Golvach at 12:05 AM  

binary, calculator, character, decimal, function, linux, octet, pack, perl, script, unix, unpack

• ▼ 2011 (1)
  • ▼ January (1)
    • Updated Post: Arbsorption Of Knowledge In The Com...

• ► 2010 (2)
  • ► December (1)
  • ► January (1)

• ► 2009 (179)
  • ► June (28)
  • ► May (31)
  • ► April (30)
  • ► March (31)
  • ► February (28)
  • ► January (31)

• ► 2008 (366)
  • ► December (31)
  • ► November (30)
  • ► October (31)
  • ► September (30)
  • ► August (31)
  • ► July (31)
  • ► June (30)
  • ► May (31)
  • ► April (30)
  • ► March (31)
  • ► February (29)
  • ► January (31)

• ► 2007 (101)
  • ► December (36)
  • ► November (36)
  • ► October (25)
  • ► September (4)