Fighting BITROT and URE with every last breath
I have always been a huge proponent of typical RAID.
Back in the day of 2TB drives, disks were quite reliable.
Even when a disk failed, I never had a problem doing a RAID recovery/resync onto a new disk.
The RAID would keep running for days or weeks no problem even with 1 disk missing,
up until the point a new disk could be inserted and replace the bad disk.
From that perspective, the RAID system was very reliable.
Fast-Forward to today where 8TB drives are ubiquitous and the probability
of doing a RAID5 recovery a 9x8TB array is between 0.3% and 56.2%.
I had to learn about URE percentages the hard way, so here we are.
I propose that the fundamentals of RAID are still good (including the software),
but doing a full recovery on a LARGE array is no longer a realistic option
given the bit error rates of current drives (i.e. RAID5 on a 72TB array).
I choose to make many small "microraids" to encapsulate my data.
This will keep the recovery percentages very high for each array.
Each microraid is backed by a set of disk images, placed anywhere on any disk.
For each microraid you can choose a different level of redundancy,
even though they are stored on the same set of physical disks.
microraids gives the flexibility to have any number of RAID 0/1/4/5/6 arrays as long as available drive space will allow it.
Also each microraid can be checked independently for integrity and consistency in multiple ways.
- Disk capacities do not have to match
- Integrity issues can be identified before recovery
- Recovery probabilities are significantly increased
- You can put multiple raid types (i.e. 0/1/4/5/6) on the same disk
- calc
- Standard Utilities: sgdisk / hdparm / dd / losetup / mdadm
- Step 1: Prepare your physical disks
- Step 2: Create your microraid
- Step 3: Start/Stop your microraid
- Step 4: Check your microraid
- Step 5: Replace a faulty image
- Step 6: Tuning your microraid (Optional)
- Step 7: Grow your microraid (Optional)
- Cooler Master Elite 110
- ASRock J3455B-ITX / J4005B-ITX / J4105B-ITX / J4025B-ITX / J4125B-ITX
- These ASRock Mini-ITX boards support PCIe in x2 mode
- Ableconn PEX-SA130
- ASM1062 chipset supports 2x Port Multiplier
- 2x Mediasonic ProBox HF2-SU3S2
- Each ProBox connects 4x 3.5" drives of any size via eSATA
- 8x 4TB HGST Ultrastar 7K4000 REFURB
- Chosen to show software reliability/recovery on very inexpensive hardware
- SYBA SI-PEX40072
- This Marvell Chipset also supports 2x Port Multiplier
- Ableconn PEX-SA134
- 2xASM1062 chips support 4x Port Multiplier; requires PCIe x4 lanes