| Chapter 15. Concatenated Disk Device (CCD) configuration | ||
|---|---|---|
| Prev | Part III. System configuration, administration and tuning | Next |
Table of Contents
The CCD driver allows the user to "concatenate" several physical disks into one pseudo volume. While RAIDframe (see Chapter 16, NetBSD RAIDframe) also allows doing this to create RAID level 0 sets, it does not allow you to do striping across disks of different geometry, which is where CCD comes in handy. CCD also allows for an "interleave" to improve disk performance with a gained space loss. This example will not cover that feature.
The steps required to setup a CCD are as follows:
Install physical media
Configure kernel support
Disklabel each volume member of the CCD
Configure the CCD conf file
Initialize the CCD device
Create a filesystem on the new CCD device
Mount the CCD filesystem
This example features a CCD setup on NetBSD/sparc 1.5. The CCD will reside on 4 SCSI disks in a generic external Sun disk pack chassis connected to the external 50 pin SCSI port.
This step is at your own discretion, depending on your platform and the hardware at your disposal.
From my DMESG:
Disk #1: probe(esp0:0:0): max sync rate 10.00MB/s sd0 at scsibus0 target 0 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed sd0: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors Disk #2 probe(esp0:1:0): max sync rate 10.00MB/s sd1 at scsibus0 target 1 lun 0: <SEAGATE, ST32430N SUN2.1G, 0444> SCSI2 0/direct fixed sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors Disk #3 probe(esp0:2:0): max sync rate 10.00MB/s sd2 at scsibus0 target 2 lun 0: <SEAGATE, ST11200N SUN1.05, 9500> SCSI2 0/direct fixed sd2: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors Disk #4 probe(esp0:3:0): max sync rate 10.00MB/s sd3 at scsibus0 target 3 lun 0: <SEAGATE, ST11200N SUN1.05, 8808 > SCSI2 0 sd3: 1005 MB, 1872 cyl, 15 head, 73 sec, 512 bytes/sect x 2059140 sectors
The following kernel configuration directive is needed to provide CCD device support. It is enabled in the GENERIC kernel:
pseudo-device ccd 4 # concatenated disk devices
In my kernel config, I also hard code SCSI ID associations
to /dev device entries to prevent bad
things from happening:
sd0 at scsibus0 target 0 lun ? # SCSI disk drives sd1 at scsibus0 target 1 lun ? # SCSI disk drives sd2 at scsibus0 target 2 lun ? # SCSI disk drives sd3 at scsibus0 target 3 lun ? # SCSI disk drives sd4 at scsibus0 target 4 lun ? # SCSI disk drives sd5 at scsibus0 target 5 lun ? # SCSI disk drives sd6 at scsibus0 target 6 lun ? # SCSI disk drives
Each member disk of the CCD will need a special file system established. In this example, I will need to disklabel:
/dev/rsd0c
/dev/rsd1c
/dev/rsd2c
/dev/rsd3c
Always remember to disklabel the character device,
not the block device, in
/dev/r{s,w}d*
On all platforms, the c slice is
symbolic of the entire NetBSD partition and is reserved.
You will probably want to remove any pre-existing disklabels on the disks in the CCD. This can be accomplished in one of two ways with the dd(1) command:
#dd if=/dev/zero of=/dev/rsd0c bs=8k count=1#dd if=/dev/zero of=/dev/rsd1c bs=8k count=1#dd if=/dev/zero of=/dev/rsd2c bs=8k count=1#dd if=/dev/zero of=/dev/rsd3c bs=8k count=1
If your port uses a MBR (Master Boot Record) to partition the disks so that the NetBSD partitions are only part of the overall disk, and other OSs like Windows or Linux use other parts, you can void the MBR and all partitions on disk by using the command:
#dd if=/dev/zero of=/dev/rsd0d bs=8k count=1#dd if=/dev/zero of=/dev/rsd1d bs=8k count=1#dd if=/dev/zero of=/dev/rsd2d bs=8k count=1#dd if=/dev/zero of=/dev/rsd3d bs=8k count=1
This will make all data on the entire disk inaccessible. Note
that the entire disk is slice d on i386
(and some other ports), and c elsewhere (e.g. on
sparc). See the "kern.rawpartition" sysctl - "3"
means "d", "2" means "c".
The default disklabel for the disk will look similar to this:
#disklabel -r sd0[...snip...] bytes/sector: 512 sectors/track: 116 tracks/cylinder: 9 sectors/cylinder: 1044 cylinders: 3992 total sectors: 4197405 [..snip...] 3 partitions: # size offset fstype [fsize bsize cpg] c: 4197405 0 unused 1024 8192 # (Cyl. 0 - 4020*)
You will need to create one "slice" on the NetBSD partition of the disk that consumes the entire partition. The slice must begin at least one cylinder offset from the beginning of the disk/partition to provide space for the special CCD disklabel. The offset should be 1x sectors/cylinder (see following note). Therefore, the "size" value should be "total sectors" minus 1x "sectors/cylinder". Edit your disklabel accordingly:
#disklabel -e sd0
The offset of a slice of type "ccd" must be a multiple of the "sectors/cylinder" value.
Be sure to export EDITOR=[path to your favorite editor] before editing the disklabels.
The slice must be fstype ccd.
Because there will only be one slice on this partition,
you can recycle the c slice (normally
reserved for symbolic uses). Change your disklabel to
the following:
3 partitions: # size offset fstype [fsize bsize cpg] c: 4196361 1044 ccd # (Cyl. 1 - 4020*)
Optionally you can setup a slice other than
c to use, simply adjust accordingly
below:
3 partitions: # size offset fstype [fsize bsize cpg] a: 4196361 1044 ccd # (Cyl. 1 - 4020*) c: 4197405 0 unused 1024 8192 # (Cyl. 0 - 4020*)
Be sure to write the label when you have completed. Disklabel will object to your disklabel and prompt you to re-edit if it does not pass its sanity checks.
Once all disks are properly labeled, you will need to
generate a configuration file,
/etc/ccd.conf. The file does not exist by
default, and you will need to create a new one. The format
is:
#ccd ileave flags component devices
For the "ileave", if a value of zero is used then the disks are concatenated, but if you use a value equal to the "sectors/track" number the disks are interleaved.
Example in this case:
#more /etc/ccd.confccd0 0 none /dev/sd0c /dev/sd1c /dev/sd2c /dev/sd3c
The CCD driver expects block device files as components. Be sure not to use character device files in the configuration.
Once you are confident that your CCD configuration is sane, you can initialize the device using the ccdconfig(8) command: Configure:
#ccdconfig -C -f /etc/ccd.conf
Unconfigure:
#ccdconfig -u -f /etc/ccd.conf
Initializing the CCD device will activate
/dev entries:
/dev/{,r}ccd#:
#ls -la /dev/{,r}ccd0*brw-r----- 1 root operator 9, 0 Apr 28 21:35 /dev/ccd0a brw-r----- 1 root operator 9, 1 Apr 28 21:35 /dev/ccd0b brw-r----- 1 root operator 9, 2 May 12 00:10 /dev/ccd0c brw-r----- 1 root operator 9, 3 Apr 28 21:35 /dev/ccd0d brw-r----- 1 root operator 9, 4 Apr 28 21:35 /dev/ccd0e brw-r----- 1 root operator 9, 5 Apr 28 21:35 /dev/ccd0f brw-r----- 1 root operator 9, 6 Apr 28 21:35 /dev/ccd0g brw-r----- 1 root operator 9, 7 Apr 28 21:35 /dev/ccd0h crw-r----- 1 root operator 23, 0 Jun 12 20:40 /dev/rccd0a crw-r----- 1 root operator 23, 1 Apr 28 21:35 /dev/rccd0b crw-r----- 1 root operator 23, 2 Jun 12 20:58 /dev/rccd0c crw-r----- 1 root operator 23, 3 Apr 28 21:35 /dev/rccd0d crw-r----- 1 root operator 23, 4 Apr 28 21:35 /dev/rccd0e crw-r----- 1 root operator 23, 5 Apr 28 21:35 /dev/rccd0f crw-r----- 1 root operator 23, 6 Apr 28 21:35 /dev/rccd0g crw-r----- 1 root operator 23, 7 Apr 28 21:35 /dev/rccd0h
You may now disklabel the new virtual disk device associated with your CCD:
#disklabel -e ccd0
Once again, there will be only one slice, so you may either
recycle the c slice or create a separate
slice for use.
#disklabel -r ccd0# /dev/rccd0c: type: ccd disk: ccd label: default label flags: bytes/sector: 512 sectors/track: 2048 tracks/cylinder: 1 sectors/cylinder: 2048 cylinders: 6107 total sectors: 12508812 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # microseconds track-to-track seek: 0 # microseconds drivedata: 0 # size offset fstype [fsize bsize cpg] c: 12508812 0 4.2BSD 1024 8192 16 # (Cyl. 0 - 6107*)
The filesystem will then need to be formatted:
#newfs /dev/rccd0cWarning: 372 sector(s) in last cylinder unallocated /dev/rccd0c: 12508812 sectors in 6108 cylinders of 1 tracks, 2048 sectors 6107.8MB in 382 cyl groups (16 c/g, 16.00MB/g, 3968 i/g) super-block backups (for fsck -b #) at: [...]
Once you have a created a file system on the CCD device,
you can then mount the file system against a mount point
on your system. Be sure to mount the slice labeled type
ffs or 4.2BSD:
#mount /dev/ccd0c /mnt
Then:
#export BLOCKSIZE=1024; dfFilesystem 1K-blocks Used Avail Capacity Mounted on /dev/sd6a 376155 320290 37057 89% / /dev/ccd0c 6058800 1 5755859 0% /mnt
Congratulations, you now have a working CCD. To configure
the CCD device at boot time, set
ccd=yes in
/etc/rc.conf. You can adjust
/etc/fstab to get the filesystem mounted at
boot:
/dev/ccd0c /home ffs rw 1 2