The Linux and Unix Menagerie

Hey There,

Today, we're going to go back a bit (we'll be putting out the final script from our number pool series this Wednesday) and take a look at a subject we've visited before in posts on making our Thesaurus script better and improving our webserver access log parser . Today, we present a fairly simple Perl script that will parse any file consisting of rows of numbers and print out matches meeting certain criteria. This is a fairly banal concept (and most probably well-overdone ;), so the spin we're going to put on it for this 2-parter is to present the Perl script (Suitable for running on any Unix or Linux distro) we've written to do this, first, in its completely "lame" format.

Now, when we say "lame," we don't mean that it doesn't do what it's supposed to; only that it is written in an overly cumbersome and confusing manner and will probably make any Perl enthusiast nauseated at the mere sight of it ;) The script is fairly simple and only requires that you have a file to parse with it. That file should be of the format:

host # cat file
01 02 03 04 05 09
05 18 19 45 33
33 55 666 88 23 12
...

host # ./match.pl
Error Encountered! Invalid or incomplete options!
Usage: ./match.pl -h highNumber -f statFile -n numberOfCombos -m mimimumCombos

host # ./match.pl -h 39 -f MYFILE -n 3 -m 2
13 29 19 matches 2 times:
 13 19 26 29 36
 13 15 19 22 29

13 30 12 matches 2 times:
 11 12 13 29 30
 05 12 13 21 30

13 31 16 matches 2 times:
 01 13 16 28 31
 13 16 20 22 31
 ...

#!/usr/bin/perl

#
# match.pl
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 
#

use Getopt::Std;

%options = ();

getopts("h:f:n:m:", \%options);

if ( defined $options{m} && defined $options{h} && defined $options{f} && defined $options{n} ) {
 $highnum = $options{h};
 $combos = $options{n};
 $statfile = $options{f};
 $minmatch = $options{m};
 if ( ! -f $statfile ) {
 usage("File $options{f} Does Not Exist!");
 } elsif ( $combos > 6 ) {
 usage("Only Combos Up To 6 Please!");
 }
} else {
 usage("Invalid or incomplete options!");
}


open(FILE, "<$statfile");
@file = <FILE>;
close(FILE);

for ( $lownum = 1; $lownum <= $highnum; $lownum++ ) {
 if ( $lownum < 10 ) {
 $padded_num = "0$lownum";
 } else {
 $padded_num = $lownum;
 }
 push(@numbers, "$padded_num");
}

for ( $times = 1; $times <= $combos; $times++) {
 if ( $combos == 1 ) {
 foreach $cnum1 (@numbers) {
 chomp($cnum);
 @match = grep(/$cnum1/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 matches $match times:\n @match\n";
 }
 }
 } elsif ( $combos == 2 ) {
 foreach $cnum1 (@numbers) {
 foreach $cnum2 (@numbers) {
 chomp($cnum);
 if ( $cnum1 != $cnum2 ) {
 @match = grep(/$cnum1/ && /$cnum2/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 $cnum2 matches $match times:\n @match\n";
 }
 }
 }
 }
 } elsif ( $combos == 3 ) {
 foreach $cnum1 (@numbers) {
 foreach $cnum2 (@numbers) {
 foreach $cnum3 (@numbers) {
 chomp($cnum);
 if ( $cnum1 != $cnum2 && $cnum1 != $cnum3 && $cnum2 != $cnum3 ) {
 @match = grep(/$cnum1/ && /$cnum2/ && /$cnum3/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 $cnum2 $cnum3 matches $match times:\n @match\n";
 }
 }
 }
 }
 }
 } elsif ( $combos == 4 ) {
 foreach $cnum1 (@numbers) {
 foreach $cnum2 (@numbers) {
 foreach $cnum3 (@numbers) {
 foreach $cnum4 (@numbers) {
 chomp($cnum);
 if ( $cnum1 != $cnum2 && $cnum1 != $cnum3 && $cnum1 != $cnum4 && $cnum2 != $cnum3 && $cnum2 != $cnum4 && $cnum3 != $cnum4 ) {
 @match = grep(/$cnum1/ && /$cnum2/ && /$cnum3/ && /$cnum4/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 $cnum2 $cnum3 $cnum4 matches $match times:\n @match\n";
 }
 }
 }
 }
 }
 } 
 } elsif ( $combos == 5 ) {
 foreach $cnum1 (@numbers) {
 foreach $cnum2 (@numbers) {
 foreach $cnum3 (@numbers) {
 foreach $cnum4 (@numbers) {
 foreach $cnum5 (@numbers) {
 chomp($cnum);
 if ( $cnum1 != $cnum2 && $cnum1 != $cnum3 && $cnum1 != $cnum4 && $cnum1 != $cnum5 && $cnum2 != $cnum3 && $cnum2 != $cnum4 && $cnum2 != $cnum5 && $cnum3 != $cnum4 && $cnum3 != $cnum5 && $cnum4 != $cnum5 ) {
 @match = grep(/$cnum1/ && /$cnum2/ && /$cnum3/ && /$cnum4/ && /$cnum5/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 $cnum2 $cnum3 $cnum4 $cnum5 matches $match times:\n @match\n";
 }
 }
 }
 }
 }
 } 
 }
 } elsif ( $combos == 6 ) {
 foreach $cnum1 (@numbers) {
 foreach $cnum2 (@numbers) {
 foreach $cnum3 (@numbers) {
 foreach $cnum4 (@numbers) {
 foreach $cnum5 (@numbers) {
 foreach $cnum6 (@numbers) { 
 chomp($cnum);
 if ( $cnum1 != $cnum2 && $cnum1 != $cnum3 && $cnum1 != $cnum4 && $cnum1 != $cnum5 && $cnum1 != $cnum6 && $cnum2 != $cnum3 && $cnum2 != $cnum4 && $cnum2 != $cnum5 && $cnum2 != $cnum6 && $cnum3 != $cnum4 && $cnum3 != $cnum5 && $cnum3 != $cnum6 && $cnum4 != $cnum5 && $cnum4 != $cnum6 && $cnum5 != $cnum6 ) {
 @match = grep(/$cnum1/ && /$cnum2/ && /$cnum3/ && /$cnum4/ && /$cnum5/ && /$cnum6/, @file);
 $match = @match;
 if ( $match >= $minmatch ) {
 print "$cnum1 $cnum2 $cnum3 $cnum4 $cnum5 $cnum6 matches $match times:\n @match\n";
 }
 }
 }
 }
 }
 } 
 }
 }
 }
}

sub usage {
 $message = shift;
 print "Error Encountered! $message\n";
 print "Usage: 0ドル -h highNumber -f statFile -n numberOfCombos -m mimimumCombos\n";
 exit(1);
}

Posted by Mike Golvach at 12:50 AM  

array, foreach, linux, perl, recursion, script, unix

Hey there,

Today, we're going to step into the wayback machine and take a look at something very basic that, if ignored or forgotten, can potentially have devastating side-effects. And, I'm not referring to the way I used to drink in college ;) This has more to do with an op-ed piece we ran a while back about paying attention to the small things . In fact, it has exactly to do with that.

The topic for today's blast of hot air is "infinite recursion." Specifically, we'll be looking at a kind of insidious way of getting stung by that issue through the use of shell aliases. They seem harmless, and they do make it so you can, for instance, type "sit-stay" instead of "while :;do print -n "*";sleep 15;done" when you want to walk away from your terminal and not disconnect your SSH session, but they do have a down side, which is fairly easy to exploit. In my experience, these cpu-killers happen most often "by accident" rather than with any malicious intent. Nevertheless, their very existence cries out for caution (Tomorrow I might write some fiction to get all these fifty cent phrases out of my system. Although some folks may contend that I've done enough of that on this blog already ;)

The infinite recursion problem with aliases closely parallels a very simple exploit that can be run in a simple shell script. The ends are the same, and the means are closely related. For instance, if you create a shell script called "ls," with the contents:

#!/bin/sh

ls;sleep 1500000000

and manage to get that in a user's PATH before the real /bin/ls (or /usr/bin/ls), it'll ratchet up the number of open processes and filehandles very quickly. If let sit, it will take down any machine of any size eventually. This end can actually be accomplished even more simply than that, but we're not trying to encourage reckless behaviour; just rubbernecking a little ;)

The problem you can get into with simple shell aliases is when you have an alias for a command (like "ls") that actually already exists. I will usually make my aliases unique (like "sit-stay," above, which substitutes for an asterisk-printing while loop), but not everyone does. And, sometimes, an alias that is unique can become the opposite if you change OS's. For instance "ll" doesn't exist on Solaris, but it does on HP-UX.

It's very important that, when you create an alias which is named the same as a system binary (or shell built-in) that you compensate for that fact. It's actually made very easy to protect yourself, from yourself, by pretty much every shell I've ever worked with. At the most basic level, you can instruct your shell to not even attempt alias expansion of whatever you type on the command line by simply enclosing it in quotes (single or double should both work - If not, let me know what shell you're using. It would be interesting to have a list of shells that make this distinction). So, while:

host # cd /tmp

would (and I'm oversimplifying this by a preposterous degree) cause the shell to check, in order, if "cd" was an alias, a shell built-in, a function, an actual binary in our PATH or an erroneous entry (stopping at whatever point returns true first), typing:

host # "cd" /tmp

would remove the "alias check" from the equation. Problem solved and/or potential for damage neutralized. Just like the simple shell script above, if you created an alias for "cd" (which might differ from this shell to yours) like this one:

host # alias cd="cd $@"

you could have a problem on your hands. Lord knows why you'd want to do this, since you'd still be typing "cd wherever" either way, but I'm just being obtuse to make a point ;) In this situation, every time someone types something like:

host # cd /my/home/dir

The shell will determine that "cd" is an alias before it processes the command and, while it's processing the alias it will see that the actual command (to which the alias refers) contains an alias, which it will process, etc, etc, and, before you know it, system resources will become depleted to the point that no new processes can be forked. The machine, however, will be completely forked ;)

The preferred way (or so they tell me) is to encapsulate your alias in a function of the type _NAME (for the command NAME) (even though you, theoretically, only have to make sure your alias has the "real" command in quotes):

function _cd {
 "cd" $@
}
alias cd="_cd"

function _cd {
 "cd $@"
 return=$?
 echo $PWD
 return $return
}
alias cd="_cd"

Posted by Mike Golvach at 12:11 AM  

alias, builtin, linux, loop, recursion, recursive, solaris, unix

Hey There,

For this "Lazy Sunday" post, we're going to take a look at a Perl script, for Linux or Unix, that checks whether or not a string you input is a palindrome. As you probably know (but I feel I should explain anyway, just to be thorough ;), a palindrome is any set of letters, numbers, spaces, etc, that reads exactly the same forward or backward.

Check out this little script and notice the use of recursion to both maintain a pseudo "state" and the use of a single subroutine to perform what could be a complex operation.

We'll port this to shell in a few days (like we've done in our post on porting a web element reporting script and its improved log checking follow up), with more explanation of what's going on, and the why and how of what gets changed when we port from Perl to shell.

Enjoy your Sunday and relax :)

Cheers,


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

#!/usr/bin/perl

#
# perlpal.sh
#
# 2008 - Mike Golvach - eggi@comcast.net
#
# Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License 
#

print "Enter A String: ";
$string=<STDIN>;
chomp($string);

$chars = length($string);
@string = split(//, $string);
$count=0;
$status = palindrome( @string, $chars, $count );


if ( $status == 1 ) {
 print "\n That String Is A Palindrome.\n\n";
} else {
 print "\n That String Is Not A Palindrome.\n\n";
}

sub palindrome(@string, $n\chars, $count) {
 if ( $string[$count] eq $string[$chars-$count-1] ) {
 $count++;
 palindrome($string, $chars, $count);
 }
 if ($count == $chars) {
 return 1;
 } else {
 return 0;
 }
}

Posted by Mike Golvach at 12:06 AM  

linux, palindrome, perl, port, porting, recursion, script, shell, subroutine, unix

Hey There,

Computer security, for Linux and Unix, grew into its own industry probably decades ago, now. It's my own opinion, after working in this arena for around 12 years or so (or is it 11? Time to check my pulse ;) that a lot of time is wasted in middle-corporate America worrying about Sarbanes Oxley and whatever security fad replaces it. I'm not arguing that there shouldn't be high standards for company's who's job it is to "secure" things; like our credit card numbers, social security information and what have you. But, then, most of those folks aren't shooting for Soxley certification; they've already got it and that may make your information safer... probably.

Most places I've passed through, or stayed at for an extended length of time, were generally "in the process" of becoming certified in "this," which was soon to be replaced by "that" certification. And God bless everyone who makes a living trying to keep our stuff as secure as possible. They provide a valuable service and should be respected, even if they do insist that we follow the rules ;)

A lot of times, in this continuous shifting of security paradigms, the little things get overlooked. Thousands of servers will get patched over the course of a year, while accounts with simple passwords (belonging to disgruntled employees who stormed off the job years ago) sit around on, sometimes, publicly accessible boxes. And there's really not too much we can do about it (aside from getting more systems administrators and Unix/Linux professionals on the payroll :) except continue to do what we can to try and keep things sane. There will always be a weakest link in any given system and, for every link, some malcontent looking to "break a link" to let off a little steam ;)

To demonstrate what I mean about the tenuousness of the security we all think we enjoy, I present the following line of code that anyone with sufficient permission (or on a machine that isn't locked down so tightly that it's barely usable) can execute. Of course, the old trick is to get someone else to execute it for you...

(ls -R / &);exec ls

Yes, that's it. That or any other command that's accessible to most users. If you put that line in an executable file called:

ls <--- Or whatever the name of the command you're calling from within your script is.

you can run up some numbers really quick. I realize this is a lame trick, but it's pretty much my policy that I don't put anything out on this blog that's seriously detrimental. Anyone who can program shell or Perl and understands the concepts of forking, exec'ing and/or infinite recursion can come up with a million ways to exploit this weakness. Sometimes it happens by accident, like when they first introduced frames in HTML (Remember that?) Whoops ;)

And here's to the gentleman/lady who, some time in the future, figures out how to make it impossible for people to do this sort of thing while leaving a system usable. As far as I know, at this point, the only protection against attacks from the inside like these is removing all user accounts, including root... and then pulling the plug and dropping a refrigerator on your server; assuming your server isn't the size of a refrigerator. In any event, if you decimate the entire city block your server resides on, that should take care of the issue ;)

I generally have faith in people, and treat them as untrustworthy only after they've shown me that they can't be trusted. I find it hard to have respect for cut-and-paste "script kiddies" who go out and wreak havoc on systems or corporations with little or no understanding of the mechanisms that underlie their reckless behaviour. I do believe that we need to keep paying people to ethically hack our systems regularly. I remember when this was standard practice (perhaps it still is, somewhere in the world I haven't work in a while ;) and it always seemed like a good idea to me. In the worst case, you were allowing someone to channel their destructive urges into something beneficial.

So keep on keeping an eye on the little things. They generally do the most damage. Weakness can be a great strength, as they say. If you find bugs, report them. Help make the world a better place by breaking things. Who could ask to be paid (if not monetarily, then with the respect and/or gratitude of your peers) for anything more fun than that :)

Cheers,

, Mike

linux unix internet technology


[フレーム]

Posted by Mike Golvach at 12:12 AM  

exec, fork, hack, linux, recursion, security, trick, unix

• ▼ 2011 (1)
  • ▼ January (1)
    • Updated Post: Arbsorption Of Knowledge In The Com...

• ► 2010 (2)
  • ► December (1)
  • ► January (1)

• ► 2009 (179)
  • ► June (28)
  • ► May (31)
  • ► April (30)
  • ► March (31)
  • ► February (28)
  • ► January (31)

• ► 2008 (366)
  • ► December (31)
  • ► November (30)
  • ► October (31)
  • ► September (30)
  • ► August (31)
  • ► July (31)
  • ► June (30)
  • ► May (31)
  • ► April (30)
  • ► March (31)
  • ► February (29)
  • ► January (31)

• ► 2007 (101)
  • ► December (36)
  • ► November (36)
  • ► October (25)
  • ► September (4)