skip to main | skip to sidebar
Showing posts with label patch. Show all posts
Showing posts with label patch. Show all posts

Wednesday, November 26, 2008

Patching Your Solaris Server From A Network Mount

Hey again,

Today we're going to take a quick look at how to patch your Solaris server from the network (not version-specific, unless you're still using 2.4 in which case I don't remember how to help you, although I'm almost positive the answer is similar ;)

The first question you might be asking yourself is "Why in the world would I ever want, or even need, to do this?" Since I'm a somewhat-honest guy who writes from the hip, I'll tell you that, depending upon where you work and what kind of setup you have there, you may never ever need to do this. If (worst case) you can get a console connection to your server (which you'll need for this method, anyway) and have onsite staff that can pop a cd in a working drive, you probably wouldn't even want to do this. So, the simple answer is: Why not? If the unlikely situation arises (and, if you're reading this in the distant future, it just might have ;), you'll know how to do it. If you set yourself up correctly, it's just one more way to get around having to come into the office in the middle of the night :)

The first thing you'll want to do is to set up a boot server or a jumpstart server. Since that topic is fairly lengthy to go over, and we've already gone over it at length, please check out this old post on begin and finish script variable definitions for Solaris' jumpstart. It's actually the last part of a series of posts on Jumpstart, but, since it wasn't posted in backward chronological order, the last link (this one) connects back to all the previous posts. You can also get a feel for some of your extended options (on both the jumpstart server and the jumpstart client) in these posts on jumpstart booting .

The second thing to do would be to ensure that you have the patch bundle you need either on the local machine, on a partition that would normally get mounted in single user mode (if you can, put them in a subdirectory of / - but only if you have the space. /usr and /var "should" always be good candidates for placement, as well, but I've done this a couple times under conditions so crippling it took me months before I could walk upright again ;). Having your patches available when you get into single user mode (with no networking) is of vital importance when you want to patch your server. Did that sound redundant? Good; because it was ;)

Now, assuming you've prepped your Jumpstart server, and have it all set up to service your client (the machine we're going to patch), it's a simple matter to get things started on your end (We'll assume that you have either a console, ALOM or other such connection to your patch client server so that you won't get disconnected from it when it goes to PROM and can interact with it at any run level or phase). The following should suffice to get the whole shebang (not to be confused with shebang ;) started:

host # id
uid=0(root) gid=0(root)
host # init 0
ok> boot net -s


And we're off. Once your server has successfully booted into single user mode on your Jumpstart, or Boot, server (see above for links to more information regarding those details), you can begin going about your business and, hopefully, finishing well before the time you promised you'd be done.

First (or is it third?), after you've booted into single user mode (no networking) and verified that you can access your patches, be sure to stop any and all unnecessary services. Most of them should be gone, but a simple "ps -ef" and "df -k" to verify that no one snuck in an init script or mount at a run level you wouldn't normally suspect might save you some headache later, and it'll probably only take about 2 or 3 minutes at most, even if you have to "terminate" a process or two. Also be sure to mount any partitions that need to be mounted if, for some reason, they don't get mounted automatically. You should get errors, when you attempt to patch, that will indicate what you need to fix if something is wrong in this regard, but it's usually best just to have things ready and not take the chance that Sun forgot to put an error-check in a script somewhere!

And the rest (much like this post) is academic. The general procedure (assuming you've downloaded a recommended patch bundle from Sun and have it in the directory /patches (or something similar), would go something like this (although possibly slightly differently if you're running Solaris 10 with zones , which case you should check out that hyperlink a few words back:

host # cd /patches
host # ./install_cluster
...
< wait and watch, or go away and come back - just make sure you see all the error messages. Most will probably be inconsequential if you're using a generic recommended patch cluster ("software that the patch patches isn't installed" or "greater revision of patch already installed" are technically errors, but can be safely ignored.
host # reboot -- -r

And, that should be that :) Remember, this is actually a much longer post, if you need to go back to the referenced posts and get details for all of the parts. I thought it would be nice to have everything in one place (or linked to one place) finally, with regards to this topic, so I wrote this patch-work wire-frame mockery of a sham ;)

Have fun patching and be sure to do it, at the very least, more often than the Cicadas hatch ;)

Cheers,

, Mike




Please note that this blog accepts comments via email only . See our Mission And Policy Statement for further details.

Posted by Mike Golvach at 12:56 AM  

, , , , , , ,

Thursday, June 26, 2008

Mirroring Your Boot Disk Using Veritas On Sun Without Encapsulation

Hey There,

Today, we're going to take a look at a using Veritas to mirror your boot disks, without using encapsulation. There's raging debate about whether or not Veritas root disk encapsulation is bad (some folks are great with it because it takes care of everything, while some folks hate it because it's very hard to deal with when it doesn't take care of everything ;)

The following procedure was introduced quite a while ago by a pair of gentleman named Gene Trantham and John S. Howard in an article they wrote for Sun Blueprints Online (A site run by Sun that highlights best-practices for many, many things Solaris). I believe it's now simply referred to as the "Trantham/Howard Method."

Although I've used this method many times in my career, there's no point in re-writing the instructions (although I did, to a certain degree, to make them more concise and accessible). I did, however, want to have this information included on my blog, to complement (and complete) our earlier posts on patching your Veritas root disk, Veritas Volume Group setup , etc. The full document can be read here in the August 2000 Online Edition of Sun Blueprints.

So, without further ado, and, hopefully you'll find this helpful as well, the "Trantham/Howard Method" for mirroring boot disks using Veritas on Sun Solaris (somewhat modified and truncated):

Enjoy,

1. Install the basic Veritas Volume Manager Software.

2. Install any required Veritas licenses.

3. Use vxinstall to encapsulate the root disk c0t0d0 (Or whatever cxtxdx disk name accurately reflects your setup). Choose not to encapsulate the root disk and don't mirror it. For this post's sake, name it rootdisk Once the root disk is encapsulated, simply reboot and check "vxprint -ht" - You should note that only your rootdisk (and its Veritas name) is listed out.

4. Initialize the root mirror disk c0t1d0 (again, substitute with your own specific controller-target-disk name).

host # /usr/lib/vxvm/bin/vxdisksetup -i c0t1d0
host # vxdg adddisk rootmirror=c0t1d0


5. Manually attach the mirror plexes to the root volumes with "vxrootmir" (manually mirror the swapvol, usr and var volumes, if necessary, as well):

host # /usr/lib/vxvm/bin/vxrootmir rootmirror
host # vxassist mirror swapvol alloc=rootmirror
host # vxassist mirror usr alloc=rootmirror
host # vxassist mirror var alloc=rootmirror


While you wait for this to finish, you can check on the progress with the vxtask command.

6. Once your root disk is mirrored, disassociate the original plexes on c0t0d0, remove them, and remove the encapsulated rootdisk from the rootdg diskgroup:

host # vxplex dis rootvol-01 swapvol-01 var-01 usr-01
host # vxdg rmdisk rootdisk


Now your operating system is entirely running off of the c0t1d0 disk.
At this point, the operating system is based just on the initialized disk c0t1d0, which you can see by looking at the output of "vxprint -ht" again.

7. Now initialize, as opposed to encapsulating, c0t0d0:

host # /usr/lib/vxvm/bin/vxdisksetup -i c0t0d0
host # vxdg adddisk rootdisk=c0t0d0


8. Now, mirror everything from the rootmirror disk back to the rootdisk:

host # /usr/lib/vxvm/bin/vxrootmir rootdisk
host # vxassist mirror swapvol alloc=rootdisk
host # vxassist mirror var alloc=rootdisk
host # vxassist mirror usr alloc=rootdisk


Again, you can use the vxtask command to view the progress of this command, which may take some time to complete. Also, this process removes the "rootdisk-B0" subdisk. This is used by Veritas to protect the boot block, but isn't necessary when you're not encapsulating your disks.

9. Using the output of "vxprint -ht", create disk slices corresponding to the Veritas subdisks. The usage for vxmksdpart command is:

vxmksdpart [-f] [-g diskgroup] subdisk sliceno [tag flags]

You won't need to run through this for the root slice since vxrootmir has already done this for you. You will need to do this for the other slices, though. But, don't use vxmksdpart just yet.

10. On the "new" root disk, first create the slices for the partitions you're going to mirror:

host # /usr/lib/vxvm/bin/vxmksdpart rootdisk-02 1 0x03 0x01
host # /usr/lib/vxvm/bin/vxmksdpart rootmirror-02 1 0x03 0x01


and so on for each required partition (again, you can get this output from "vxprint -ht"). The "sliceno" part of the command syntax is very important here!

host # /usr/lib/vxvm/bin/vxmksdpart rootdisk-03 5 0x07 0x00
host # /usr/lib/vxvm/bin/vxmksdpart rootmirror-03 5 0x07 0x00
...


11. Among other things (like backing up /etc/vfstab, etc) be sure to capture the current disk partition information for both disks, before rebooting, using vxprtvtoc:

host # /usr/lib/vxvm/bin/vxprtvtoc -f /tmp/disk1 /dev/rdsk/c0t0d0s2
host # /usr/lib/vxvm/bin/vxprtvtoc -f /tmp/disk2 /dev/rdsk/c0t1d0s2
host # cat /tmp/disk1 /tmp/disk2> /etc/vtoc


12. Update your system dump configuration to use the swap partition on the primary disk:

host # dumpadm -d /dev/dsk/c0t0d0s1
Dump content: kernel pages
Dump device: /dev/dsk/c0t0d0s1 (swap)
Savecore directory: /var/crash/host
Savecore enabled: yes


13. In the OBP, be sure to setup both disks using devalias (name them disk and rootdisk, if you want, or whatever makes the most sense to you) and set your configuration to have the boot-device set up to make it easy for your system to fail over if the primary disk goes bad:

ok> setenv boot-device disk, rootdisk, net

14. Be sure to test booting from the OBP for both disk and rootdisk. Assuming that goes well, you're all set :)

, Mike

Thursday, June 19, 2008

Basic Installation Security Guidelines For Linux and Unix - Part 1a

Hey there,

Today we're back at the concept-well, and starting out a five part post on setting up a Unix or Linux system securely. This should be an interesting series of posts since I (like most folks, probably) haven't worked with every flavour of Linux or Unix out there, so this set of guidelines is more of a "rough draft" based on experience with Solaris and RedHat servers than an attempt to encompass all areas of system installation, setup and security on every possible OS. Doing so would either be incredibly vague or unbelievably presumptuous. Probably, both ;)

With that in mind, we'll get going with part 1a. Setting up the basic system and keeping it clean from the start. If you're a big fan of numbered lists, you're gonna love this post ;) Hopefully, this post (and the following to come) will generate feedback from users of other systems - or these same systems running different versions - and become a nice broad-but-useful guide to system setup and security!

Best wishes,

OS Installation Guidelines, Part 1a: Basic Setup and System Security Steps (1 of 2... as in too many to fit on one reasonably sized page ;)

1. Create a list of functions the machine will be expected to perform and the services required to fulfill those needs. Also create a map of partitions you expect to use on the disks you have available.

2. Install the basic Operating System. NOTE: If you will be using disk management software (e.g. Solstice Disk Suite, Volume Manger, LVM, Veritas Volume Manager, etc), be sure to read the documentation for these packages before doing your OS install. Most of these types of packages require free partitions to store their disk management data.

3. Download the latest Recommended Patch bundle from sunsolve.sun.com for Solaris machines. Also download the Security Patch bundle if this exists separately. Install both and reboot the machine. For Redhat, do a complete up2date run-and-reboot cycle as many times as necessary.

4. Comment out the following files in /etc/init.d (Some filenames may vary or not exist, depending on what OS you're installing. For an easy way to comment out base init.d files, assuming they're linked to run level directories, check out our old post on editing startup scripts with the vi editor . Don't let the Solaris brand name in the post put you off. It works for any OS on which you can run vi) - Note, also, that this list is pulled from a Solaris box and your security standards may require removing even more. As a general rule, if you don't need it, disable it! RedHat users can do this with chkconfig or ntsysv more easily. Solaris 10 user can make use of the svcs and svcadm commands):

a. acct ( since we prefer to control the process accounting ourselves).
b. asppp ( if we won’t be hooking up a modem to do ppp dialups to the machine).
c. autofs (if automounting not necessary)
d. autoinstall (if machine is not a bootp client)
e. cachefs.daemon (if you will not be using nfs’s cachefs subsystem)
f. cachefs.root (if you will not be using nfs’s cachefs subsystem)
g. cacheos (if you will not be using nfs’s cachefs subsystem)
h. cacheos.finish (if you will not be using nfs’s cachefs subsystem)
i. dhcp (if machine is not a dhcp server or client)
j. dtlogin (if software in use does not require X-windows default login - Removing this will not remove X-windows functionality)
k. init.dmi (if machine will not be availing itself of snmp management)
l. init.snmpdx (if machine will not be availing itself of snmp management)
m. initpcmcia ( if the machine has no pcmcia devices).
n. lp (if print services are not required)
o. mkdtab ( this script runs after installation and is never used again).
p. nfs.client (if the nfs client subsystem will not be required)
q. nfs.server (if the nfs server subsystem will not be required)
r. pcmcia (if machine has no pcmcia devices)
s. power (if machine doesn’t use individual ups power management)
t. rpc (if machine will not be using rpc services) (If the machine will be using rpc services, comment out all of the start section of the script, except for the rpcbind initialization. All other parts of this section initiate nis services, which aren't necessary if you don't use nis :)
u. sendmail (if machine will not need to be a mail receiving agent.)
v. sshd (if you'll be installing you own ssh to maintain flexibility to be more up to date with any security releases).
w. spc ( if print services are not required).
x. uucp ( if you won’t be making use of the uucp subsystem).
y. xntpd (if machine will not serve as a time server or be doing persistent ntp scanning)
z. PRESERVE (this invokes /usr/lib/expreserve, which has a known buffer-overflow bug).
aa. README (read it first, if you like).

5. rm /etc/init.d/*.old

6. Check in /etc/rc0.d, /etc/rc1.d, /etc/rc2.d and /etc/rc3.d (or your system specific run level directories) for any old links to nonexistent files. Delete these.

7. Check in the rc directories, also, for files beginning with X or lowercase s or k. Delete these files as well.

8. If sshd is running, stop it. If you don't plan on using Sun's or RedHat's packaged ssh, we'll be replacing this later. You can leave it if you like.

9. Run /etc/init.d/openssh (or /etc/init.d/sshd) start to start OpenSSH.

10. Connect to the machine via ssh to continue executing the steps in this document. This way you won’t have to worry about poor console terminal emulation creating any accidental errors in the files you will be editing.

11. After installation, ensure that all unnecessary processes have been disabled, using a simple scan of ps output, and disable any that are running and not necessary (sendmail, for instance, is a program that is notorious, on Solaris, for being reactivated as a mail sending agent after patching and/or updating!)

12. Next, be sure to check the network interface. Many of the Sun machines come with all the network ndd settings enabled, which can cause problems with some routers, which are pinned to certain speeds and don’t auto-negotiate. RedHat user's should have been asked about what network settings they wanted to enforce during installation. If not, ethtool can be used to force the required speed/duplex setting Do the following to test:

i. ftp, or scp a file from somewhere, or to somewhere over the network

ii. Run netstat –ian and check the Collisions column. This should still be at 0 for the network interface. If it isn’t, you’ll need to augment the /etc/init.d/nddconfig script, use nddconfig or (for RedHat) use ethtool (and double check the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-NICNAME files) to suit the machine and/or environment.

iii. Should you need to fix the interface, so that you get zero collisions (which collisions should, theoretically be impossible on a packet-switched network) and ensure maximum speed without any dropped packets, refer to your system's, or department's documentation. There are really no generalities that can be put to paper here. In my experience, Solaris Gig NIC's usually require auto-negotiation at the switch (for Cisco's) and adv_autoneg_cap set to 1, with all other configurable parameters set to 0, in order for them to work, while old hme interfaces require that the switch be pinned to 100mb and only the adv_100fdx_cap be set to 1 in order for them to function properly. Cut and paste the contents into /etc/init.d/nddconfig after the case/esac clause and before the exit command to ensure the configuration happens at boot time. For RedHat, update the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-NICNAME files if necessary). If hme0 gives you a really big headache on Solaris, check our older post on disabling network devices at the PROM level.

13. Comment out every unnecessary line in /etc/inetd.conf (or delete or set to "off" all unnecessary files in /etc/xinetd.d), particularly the unnecessary r-services, and ensure that all services, where applicable, are managed through tcpd – ensure that telnetd is enabled as a backup during install (but, only if you plan on using a custom SSH later, and then just in case)

Ex: ftp with: /usr/local/encap/tcp-wrappers/bin/tcpd and /usr/sbin/in.ftpd –d as the file and args.

14. Add loghost to /etc/syslog.conf: (NOTE: It is imperative that a tab, and not a space, separate the list of facilities-severities and the hostname of the remote logging host or target file!) Also Note that, at this point, if you do NOT have a syslogging server dedicated and set up, you do NOT have to complete this step (not at all, if you don't plan on using one):

*.err;*.emerg;*.crit;*.alert;mail.debug;kern.notice;auth.notice;auth.info @loghost.xyz.com
*.err;kern.debug;daemon.notice;mail.crit;auth.notice;auth.info /var/adm/messages


Also be sure to change any references to /var/console to /var/adm/console on Solaris or /dev/console on RedHat.

15. Configure sendmail to use its relay properly, if you are going to use it to send mail out. If the machine is an email server; that’s another story. NOTE: If running sendmail v8.8.8 or earlier, be sure to reverse the ruleset for S94 to ensure proper masquerading of envelope addresses from the host, if you have that feature enabled!

16. Set up networking and name services minimally:

a. Create /etc/hosts with loopback, host, logging host , default router, nameserver, and mail host information if applicable. (Note: When entering host information, ensure that the first entry in the host field is the fully qualified domain name. This is a pet peeve of mine. You can do the simple hostname first and then the FQDN. Preferably, list them both for each IP.)

b. Create and populate /etc/defaultrouter or set the DEFAULT_GATEWAY value in either of the /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-NICNAME files on RedHat.

c. Edit /etc/nsswitch.conf to use "dns [ NOTFOUND="continue"]" for hosts. If you don't set NOTFOUND, the default value is "return" and this can cause issues with telnet and ftp hostname lookups, etc.

d. Create and populate /etc/resolv.conf.

17. If host is multi-homed, disable IP-forwarding by creating /etc/notrouter on Solaris. Do "echo 0> /proc/sys/net/ipv4/ip_forward" on RedHat to disable packet routing/forwarding.

18. If possible, trim /etc/mail/aliases to include at least (and, maybe, at most) the following (be sure to execute newaliases if any changes are made):

Postmaster: root
MAILER-DAEMON: root
nobody: /dev/null
root: itopsgroup@xyz.com
itopsgroup: itopsgroup@xyz.com


19. Create /etc/shells with these contents (remove any shells you don't want to be run on your system and add any, that I forgot to include, that you do!):

/sbin/sh
/bin/ash
/bin/sh
/bin/csh
/bin/ksh
/usr/bin/ksh
/bin/false
/bin/bash
/bin/zsh
/usr/local/bin/keysh (Only include if using keysh on system)


To be continued...

, Mike

Saturday, April 12, 2008

Patching Solaris 10 Zones - Global And Local Issues

Hello Again,

In a long overdue continuation of our coverage of Solaris 10 (as in our previous post on migrating between local and global zones ) today, we're going to take a look at patching on Solaris 10, insofar as it relates to zones. There are a lot of questions going around about how to patch appropriately and what's permissible. This is completely understandable since, if you do it wrong, the consequences can be disastrous and irreversible (and none of us wants to stay up all night... working ;)

We'll start off with the concept of patching and zones. As far as patching is concerned, each zone actually has it's own patch (and package) database. This makes it possible for you to, theoretically, patch single zones on a host individually or patch all of them at once by patching the "global" zone.

There are a few things to keep in mind though, before you go ahead and apply your patches, either way...

1. Even if you have "umpteen" number of zones on a single machine, they all run off of the same kernel, so if you have to patch your kernel (or anything kernel-related) you need to do that from the "global" zone. If you apply a kernel patch on a local zone, and bring its kernel patch revision to a different level than the "global" zone's (and other zones') calamity will ensue (Let's see how many different ways I can write that something bad is going to happen ;)

2. "patchadd" will eventually drive you nuts, anyway, so the following proviso's should be no surprise ;).

a. If you run patchadd with -G in a "global" zone, any packages that have SUNW_PKG_ALLZONES set to true will cause the entire patch operation to fail.

b. If you run patchadd with -G in a "global" zone and "no" packages have SUNW_PKG_ALLZONES set to true, you should be able to install the patch to all zones from the "global" zone.

c. If you run patchadd without -G in a "global" zone, regardless of the setting of SUNW_PKG_ALLZONES, you can install the appropriate patches to any individual zones or the global zone (which will patch all the zones, by default).

d. If you run patchadd in a "local" zone, with or without -G specified, any packages that have SUNW_PKG_ALLZONES set to true will fail and not install, and if none have SUNW_PKG_ALLZONES set to true, everything should work in each "local" zone you apply the patch to.

3. Any software that can be installed at the "local" zone level, can also be patched independently at the "local" zone level, on whatever zones it was installed. This is true regardless of your zone type (whole root or sparse root).

4. The "-G" option to patchadd doesn't stand for "global zone," rather it stands for "the current zone." I can't think of a good mnemonic for this so "G"ood Luck ;) Of course, if you use this flag in the "global" zone, you can pretend it makes sense and stands for that. I do...

5. The "-t" option to patchadd is available for people who got used to the old patch error code numbers and know them by heart (like me. Well... most of them). Even on a system with zones, they make it so only a return code of "0" indicates absolute success. Any other number indicates a problem.

6. The "-R" option to patchadd (with zones enabled) cannot be used to reference the root filesystem in any zone other than the "global" zone. If you choose to ignore this warning and use it on a "local" zone anyway, side effects may include damage to the "local" zone's filesystem, damage to the "global" zone's filesystem, security problems with the "global" zone's filesystem, nauseau, fatigue, dry-eye, constipation, arthritis and random "night terrors." Contact your doctor if you have any trouble breathing calmly while patching "global" zones, as this may be the sign of a rare, but serious, side effect ;)

Happy patching. Here's to eventually being allowed to do it on the weekdays :)

, Mike




[フレーム]

Sunday, March 30, 2008

Linux Update Patch For Expect User Removal Script

Hey There,

Today we have the "Linux Additions" patch to follow up on yesterday's post on removing old users network wide.

Note that this patch was created with "diff -c," so I've been sure to specify both the file I want to patch and the patch file in my arguments to "patch" (whereas I didn't need to in our post on easily patching multiple files ). The exact command line to create the patch was:

host # diff -c eraser eraser.new > eraser.patch

All you'll need to do to update yesterday's script to include today's modifications is put the patch file (tagged on to the end of this post) in the same directory as the original script (or wherever you want, assuming you know how this all works and my explaining it is trivial ;) and name it something like "eraser.patch" - Then do the following:

host # ls
. .. eraser eraser.patch
host # cp eraser eraser.bak
<--- Just in case something goes wrong, I'm always for making backups :)

As I noted before, you'll want to be specific in your arguments to "patch" when you run this, or be careful of "patch"'s default behaviour and don't choose to do a reverse patching, like this (BTW, even though my new script "eraser.new" is listed in the patch file, you don't actually need to have it for all of this to work):

host # patch -p0 -i eraser.patch
Hmm... Looks like a new-style context diff to me...
The text leading up to this was:
--------------------------
|*** eraser Sat Mar 29 18:57:29 2008
|--- eraser.new Sat Mar 29 18:57:50 2008
--------------------------
Patching file eraser using Plan A...
Reversed (or previously applied) patch detected! Assume -R? [y]
^C


Instead, patch it like this, so you can do it hands off:

host # patch -p0 eraser eraser.patch
Hmm... Looks like a new-style context diff to me...
The text leading up to this was:
--------------------------
|*** eraser Sat Mar 29 18:57:29 2008
|--- eraser.new Sat Mar 29 18:57:50 2008
--------------------------
Patching file eraser using Plan A...
Hunk #1 succeeded at 3.
Hunk #2 succeeded at 36.
Hunk #3 succeeded at 72.
Hunk #4 succeeded at 145.
Hunk #5 succeeded at 235.
Hunk #6 succeeded at 267.
done


And "diff -c" will show that your "eraser" Expect script is now different than the original that you copied off before:

host # diff -c eraser eraser.bak
*** eraser Sat Mar 29 19:02:21 2008
--- eraser.bak Sat Mar 29 18:57:50 2008
***************
...


And, of course, you can just check your "eraser" file and see the changes. If you need to back out the patch, simply do the "reverse patch" operation, like so:

host # patch -R -p0 eraser eraser.patch
Hmm... Looks like a new-style context diff to me...
The text leading up to this was:
--------------------------
|*** eraser Sat Mar 29 18:57:29 2008
|--- eraser.new Sat Mar 29 18:57:50 2008
--------------------------
Patching file eraser using Plan A...
Hunk #1 succeeded at 3.
Hunk #2 succeeded at 36.
Hunk #3 succeeded at 70.
Hunk #4 succeeded at 130.
Hunk #5 succeeded at 186.
Hunk #6 succeeded at 217.
done


And then do a cursory check to make sure the unpatched file matches the original that you copied off before:

host # diff -c eraser eraser.bak <--- You'll get no output if the files are the same. That's good :)

Enjoy and have a restful Sunday :)


Creative Commons License


This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License

*** eraser Sat Mar 29 18:57:29 2008
--- eraser.new Sat Mar 29 18:57:50 2008
***************
*** 3,9 ****
log_user 0

#########################################################################
! # eraser - root out ex-employees across network
# 2008 - Mike Golvach - eggi@comcast.net
# <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License</a>
#
--- 3,9 ----
log_user 0

#########################################################################
! # eraser - root out ex-employees across network - with Linux procs added
# 2008 - Mike Golvach - eggi@comcast.net
# <a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/3.0/us/">Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License</a>
#
***************
*** 36,41 ****
--- 36,43 ----
puts " sun1 sun2"
puts "HP Hosts:"
puts " hp1 hp2"
+ puts "LINUX Hosts:"
+ puts " linux1 linux2"
puts "#############################################################################"
puts "FLAGS:"
puts " -h: Show this message."
***************
*** 70,75 ****
--- 72,90 ----
expect "word: " {send "$qpass\r"}
}

+ proc linux_login {name tprompt login lpass} {
+ send_user "$name\n"
+
+ set timeout 3
+ set hostname $name
+ set prompt $tprompt
+ set qlogin $login
+ set qpass $lpass
+
+ expect "ogin: " {send "$qlogin\r"}
+ expect "word: " {send "$qpass\r"}
+ }
+
proc hp_login {tprompt login lpass} {

set timeout 12
***************
*** 130,135 ****
--- 145,184 ----
}
return "$hostname $insidepwd $insidealias"
}
+
+ proc linux_q {tprompt login lpass userlogin host} {
+ set timeout 5
+ set log $login
+ set opass $lpass
+ set prompt $tprompt
+ set ulogin $userlogin
+ set hostname $host
+ set insidealias none
+ set insidepwd none
+
+ send_user "Scanning $hostname... "
+
+ expect -re $tprompt {send "grep $ulogin /etc/passwd\r"}
+ expect -re $tprompt
+ send "echo $?\r"
+ expect -re "\r\n(.*)\r\n"
+ set returnval $expect_out(1,string)
+ if { $returnval == 0 } {
+ set insidepwd 1
+ }
+ expect -re $tprompt {send "grep $ulogin /etc/mail/aliases\r"}
+ expect -re $tprompt
+ send "echo $?\r"
+ expect -re "\r\n(.*)\r\n"
+ set returnval $expect_out(1,string)
+ if { $returnval == 0 } {
+ set insidealias 1
+ }
+ expect -re $tprompt {
+ send "exit\r"
+ }
+ return "$hostname $insidepwd $insidealias"
+ }

proc sco_q {tprompt login lpass userlogin host} {
set timeout 5
***************
*** 186,191 ****
--- 235,241 ----
set sun [list sun1 sun2]
set sco [list sco1 sco2]
set hp [list hp1 hp2]
+ set linux [list linux1 linux2]

set userlogin [lindex $argv 0]
set login [lindex $argv 1]
***************
*** 217,222 ****
--- 267,278 ----
spawn telnet $host
hp_login $tprompt $login $lpass
lappend finalists [sunhp_q $tprompt $login $lpass $userlogin $host]
+ }
+
+ foreach host $linux {
+ spawn telnet $host
+ linux_login $tprompt $login $lpass
+ lappend finalists [linux_q $tprompt $login $lpass $userlogin $host]
}

foreach host $sco {


, Mike




[フレーム]

Friday, March 28, 2008

Easy Multiple File Patching And Patch Removal On Linux And Unix

Hello again,

Today we're going to continue with yesterday's post on manually mass patching files on Linux and Unix, but come at it from a different, and much simpler, angle.

First things first; you can delete the ed patch files from yesterday :) When you look at them, you can see that using "diff -e" to create an ed patch file creates a patch that contains absolutely no information that would allow you to reverse a change that you made, as opposed to a patch file made from diff straight-up. The small comparison below points this up more clearly:

host #diff -e tmpdir1/file1 tmpdir2/file1
5c
BASEDIR="/usr/binky"
.
host # diff tmpdir1/file1 tmpdir2/file1
5c5
< HOMEDIR="/usr/binky"
---
> BASEDIR="/usr/binky"


The ed patch file type that we used yesterday, although simpler to understand, doesn't make it possible to reverse your changes without keeping backup copies of your files.

The good news is that patch files created with diff can still be used to patch massive amounts of files and recover them even if the original, unpatched, files get lost or deleted. The method used to obtain these results is simpler than what we walked through yesterday and only requires the use of the "diff" and "patch" commands. Both of these commands should come standard with your OS. They've been on Linux for a long time and have been on Solaris Unix since, at least, release 2.6 (probably earlier).

So, let's get started patching lots of files and then backing out those patches. Since yesterday's post was so long (and this one has the same potential), I'm going to use only 2 files as the base number of files (although the number of files can be however large you want) and grep out the relevant information for display purposes. Hopefully it will save us all some eye strain ;)

The first thing we'll want to do is copy all the scripts we want to patch into a new directory to work on them. We'll also put the new scripts (that we'll need to create our diff patches) in yet another directory. We'll never work directly on the scripts in their native directory until we're sure they're patched correctly. This isn't absolutely necessary, but is generally good practice. No sense in letting a simple mistake cost you any more time, or grief, than it has to. Again, the only thing that is different between our existing scripts and the new scripts is that the HOMEDIR variable has been changed to BASEDIR.

host # cp scriptdir/* tmpdir1/
<--- This could be any number of files.

The following is our setup, with only one difference between the files, as noted above:

host # ls tmpdir1
. .. file1 file2
host # ls tmpdir2
. .. file1 file2
host # grep DIR tmpdir1/*
tmpdir1/file1:HOMEDIR="/usr/binky"
tmpdir1/file2:HOMEDIR="/usr/binky"
host # grep DIR tmpdir2/*
tmpdir2/file1:BASEDIR="/usr/binky"
tmpdir2/file2:BASEDIR="/usr/binky"


Next, we'll use diff to create a patch file. Although, rather than just doing diff straight-up, we're going to run it in "contextual" mode. When you invoke "diff -c" it creates a contextual diff which, literally, means that it puts the diff output in context (so you can see the lines before and after the lines that differ). The main reason I like to use this option is that the output works with "patch" when patching multiple files from multiple directories. The output from a standard diff of multiple files in multiple directories doesn't work well for this (mostly because it puts all the file names on one line and "patch" attempts to find a file named "diff tmpdir1/file tmpdir2/file" - literally. And that file can never exist (I hope ;)

Now we'll create the multiple file patch and examine its contents, created at the directory level directly above tmpdir1 and tmpdir2, so we can get all the files in both directories (Note that only the lines beginning with the exclamation point (!) in the diff output are different. The lines above and below only serve to showcase the line in its context within the file):

host # diff -c tmpdir1 tmpdir2 >patchfile.patch
host # cat patchfile.patch
diff -c tmpdir1/file1 vtmpdir2/file1
*** tmpdir1/file1 Wed Mar 26 14:36:05 2008
--- tmpdir2/file1 Wed Mar 26 14:54:10 2008
***************
*** 2,8 ****

COMMAND_ARGS="-d --takeforeverandaday"

! HOMEDIR="/usr/binky"

case $x in

--- 2,8 ----

COMMAND_ARGS="-d --takeforeverandaday"

! BASEDIR="/usr/binky"

case $x in

diff -c tmpdir1/file2 vtmpdir2/file2
*** tmpdir1/file2 Wed Mar 26 14:36:05 2008
--- tmpdir2/file2 Wed Mar 26 14:54:10 2008
***************
*** 2,8 ****

COMMAND_ARGS="-d --takeforeverandaday"

! HOMEDIR="/usr/binky"

case $x in

--- 2,8 ----

COMMAND_ARGS="-d --takeforeverandaday"

! BASEDIR="/usr/binky"

case $x in


Now we're ready to patch all of the files in tmpdir1 at once, using the simple form of the command (patch -i PATCHFILE), and receive an error for doing so. Note that we're running this from the directory above tmpdir1 and tmpdir2; exactly where we were when we used "diff -c" to create the patch file:

host # patch -i patchfile.patch
can't find file to patch at input line 4
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|diff -c tmpdir1/file1 tmpdir2/file1
|*** tmpdir1/file1 Wed Mar 26 14:36:05 2008
|--- tmpdir2/file1 Wed Mar 26 14:54:10 2008
--------------------------
File to patch: ^C
<--- Type the control (ctl) key + C, or any other escape/control key combination to break out of this prompt

This error killed us before it actually made any changes because of the way "patch" works. When run without any options (other than -i, which we used to indicate the name of our patch file), "patch" parses the patch file and strips the file names down to the base, in much the same way the "basename" command does. So, even though the file name in the patch file is "tmpdir/file1," the "patch" program is looking for a file named "file1" and it's looking for it in the directory we're in which, unfortunately, isn't where the file is.

Luckily, this little setback is easy to remedy. Using the -p option to "patch" we can instruct "patch" how to interpret the file names in our patch file. As we noted, when the option isn't present, "patch" reverts to "basename" type behaviour. If we used -p1, we would be instructing "patch" to remove the leading slash (/) from the file name. We're going to use -p0, which instructs "patch" to not interpret the file name and just take it as it is (In this case "tmpdir/file1," which is relative, but just fine considering where we're running the command from).

host # patch -p0 -i patchfile.patch
patching file tmpdir1/file1
patching file tmpdir1/file2


Success! Now, let's check that the patch actually took:

host # grep DIR tmpdir1/*
tmpdir1/file1:BASEDIR="/usr/binky"
tmpdir1/file2:BASEDIR="/usr/binky"


Excellent! HOMEDIR is now BASEDIR. Alas, as I intimated in our post yesterday on mass file updating, our boss has, only minutes later, decided that the BASEDIR variable really should be HOMEDIR after all. He's not going to explain why, we just need to switch everything back now ;)

And this is where our method of execution really pays off. Assuming we held on to that patch file, we can now use "patch" to put everything back the way it was in short order, by simply adding the -R (to reverse the patch operations) to the command line and running it again, like so:

host # patch -p0 -R -i patchfile.patch
patching file tmpdir1/file1
patching file tmpdir1/file2


And, then, just to verify that the patch differences have been removed:

host # grep DIR tmpdir1/*
tmpdir1/file1:HOMEDIR="/usr/binky"
tmpdir1/file2:HOMEDIR="/usr/binky"


And we're all set :) Hopefully this follow-up tutorial was easy enough to follow, and you can find some good use for it in your work routine. BTW, don't forget to copy the changed scripts back to the real script directory, but only after copying that directory off somewhere else, again. But only if you're as paranoid as I am ;)

Best Wishes,

, Mike




[フレーム]

Posted by Mike Golvach at 12:09 AM  

, , , , , , , , ,

Thursday, March 27, 2008

Manual Mass File Patching On Linux Or Unix

Good Day Good People :)

In previous posts we've looked at various aspects of the diff command, including augmenting it to generate unique content only and work with users, groups and permissions .

Today, I thought we'd look at diff's usage in a, generally, manual process and peek at the first step toward automation. Everything in this post can be scripted out to provide more granular control, and done much more easily. I thought doing it in a laborious tutorial style to start out with would be best to get the major parts of the process laid out simply and, hopefully, written in such a way that they're simply understood. After reading that last sentence, I'm wishing myself lots of luck ;)

For purposes of our example today, we're going to assume that we have a directory in which we keep 3 executable files. All of these files are approximately the same, because they all do approximately the same thing, just for different programs. For simplicity, they've been named: file1, file2 and file3, as shown in this directory listing:

host # ls
. .. file1 file2 file3


All of their contents are also almost totally identical:

host # cat file1 file2 file3
#!/bin/bash
# Start File1 Process

COMMAND_ARGS="-d --takeforeverandaday"

HOMEDIR="/usr/binky"

#!/bin/bash
# Start File2 Process

COMMAND_ARGS="-d --takeforeverandaday"

HOMEDIR="/usr/binky"

#!/bin/bash
# Start File3 Process

COMMAND_ARGS="-d --takeforeverandaday"

HOMEDIR="/usr/binky"


For today (and this would work if we had 5000 files, but this post is going to be long enough ;) all we need to do is change the HOMEDIR standard variable in all of our scripts to BASEDIR. I'm not exactly sure why. Someone with more authority than me or you decided it would be a good idea. They will probably change their mind and have us switch it back next week ;)

The first thing we should do is create a backup directory and do our work there.

host # mkdir ../u
host # cp file* ../u
host # cd ../u
host # ls
. .. file1 file2 file3


Then we'll just use a simple while loop and feed each file to sed and make the substitution, which we'll dump into another file called FILENAME.diff (So, file1 will get the substitution made on it by sed, and that difference will be put into a new file called file1.diff). Within that same loop, we'll be using the actual command "diff -e" to create an "ed" (A very basic Unix and Linux editor) patch file. Note that, for file1, this new file will be called file1.patch and that we echo a "w" and a "q" (on separate lines on purpose) into the bottom of the patch file.

host # ls -1d *|while read x;do sed s'/HOMEDIR=\"\(\/usr\/binky\)\"/BASEDIR=\"1円\"/' $x >$x.diff;diff -e $x $x.diff >$x.patch;echo w >>$x.patch;echo q >>$x.patch;done


We could have deleted our file*.diff files above, as well, but I thought you might like to take a look at them. These serve as proof that our sed command actually worked :) HOMEDIR is now BASEDIR!

host # cat file1.diff file2.diff file3.diff
#!/bin/bash
# Start File1 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"

#!/bin/bash
# Start File2 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"

#!/bin/bash
# Start File3 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"


Now, let's delete the *.diff files and take a look at the patch files. These are all considered "ed" scripts. This means that we can feed them to the "ed" Linux/Unix editor and have it execute commands for us automatically. This will come in handy in a second :)

host # rm *.diff
host # cat file1.patch file2.patch file3.patch
5c <--- These first three
BASEDIR="/usr/binky" <--- lines were created
. <--- by "diff -e"
w <--- We manually added
q <--- these last two!
5c
BASEDIR="/usr/binky"
.
w
q
5c
BASEDIR="/usr/binky"
.
w
q


The next step in the process is to apply those patches. We're going to be doing this all in our test directory, so there's no need to tar up all the patch files for a move to our "live" script directory

host # ls
. file1 file2 file3
.. file1.patch file2.patch file3.patch


So, now we're ready to test our mass change and pray all goes well. Note that, since we're still in our backup directory, we can do lots of damage to the scripts in here and it shouldn't make a difference to anyone. Only we will have to live with the shame ;)

Now, we'll do another fancy command line and pipe the output of "ls -1d *", grepping out all the patch files and then using "ed" to patch each file in the directory. Notice that the syntax for patching a file with an "ed" diff script is "ed FILENAME <FILENAME.patch" - This would normally hang and wait for us to send it a control-D or some other signal, but, since we echoed the "w" and "q" characters into the patch files, "ed" knows to write (w) and quit (q) - At the end of the command pipe, we'll purposefully not delete all the patch files. You can remove them if you like, but we'll be looking at using them to undo changes in a future post!

host # ls -1d *|grep -v patch|while read x;do ed $x <$x.patch;done
107
107
107
107
107
107
host # ls
. file1 file2 file3
.. file1.patch file2.patch file3.patch


Okay, we're all done and now we can check the results. Looks like everything in our (potentially huge amount of) files got patched appropriately :) This shouldn't be a surprise since we checked our sed output already, but you never know.

host # cat file1 file2 file3

#!/bin/bash
# Start File1 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"

#!/bin/bash
# Start File2 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"

#!/bin/bash
# Start File3 Process

COMMAND_ARGS="-d --takeforeverandaday"

BASEDIR="/usr/binky"


In a future post we'll look at a script that will allow us to do what we've just done today and make life a little easier when doing mass updates or patching :)

Cheers,

, Mike




[フレーム]

Subscribe to: Comments (Atom)
 

AltStyle によって変換されたページ (->オリジナル) /