Storage: Efficiency measures

In 2020 we can expect bigger disk drives and hence Petabyte stores. Price per bit will come at a premium, it won't track capacity as it does now: larger capacity drives will cost more per unit.

What are the theoretical limits on which Storage solution "efficiency" can be judged?

We're slowly approaching what could be the last factor-10 improvement, to 10Tbits/in², in rotational 2-D magnetic recording technologies of Hard Disk Drives. Jim Gray (~2000) and Mark Kryder (2009) suggested 7TB/platter for 2.5" disk drives by 2020, assuming a 40%/yr capacity growth.

Rosenthal et al (2012) suggest that, like CPU-speed "Moore's Law", disk capacity growth rates have slowed, suggesting 100Tbits/in² may be possible in the far future. They predict 1.8 Tbits/in² commercially available in 2020, vs 0-6-0.7Tb/in² currently.

Three platter 2.5" drives are normally 12.5mm thick, but are in 9.5mm drives available in 2013 (HGST, 1.5TB). Four platter 2.5" drives are 12.5mm or 15mm usually, according to Seagate, with three 667GB/platter in 9.5mm for 2TB total (using 2.3W for read/write).

Slim-line 7mm and 5mm 2.5" drives are on the market. 7mm drives are two platter.

In 2020, the 2.5" disk drive market will differentiate by both thickness (5, 7, 9.5, 12.5,15mm) and number of platters, from 1 to 4. Laptop and ultrabook manufacturers will determine if 7mm replaces 9.5mm as the standard consumer portable form factor, giving them a volume production price advantage.

Per-platter, we can expect 1.5TB-2TB, or total 1TB-6TB in 2.5" drives [vs 5 platter 3.5" drives at 15TB].

Storage system builders will be able to select drive combinations on, not just SSD + HDD:

• Cost per GB
• GB per cubic-inch
• Watts per GB, and
• spindles per TB, setting maximum IO/sec and streaming IO performance.

Questions:

• How many drives can fit in a single rack?
• How much raw capacity?
• How much power would they use? [and how much cooling]
• How much does it all weight? [can the floor hold it up?]
• Time to back it up?
• Dependant on external ports and interface speeds.
• Performance:
• How many IO/sec?
• Aggregate internal streaming throughput?
• Normalised multi-media transactions/sec: 1MB Object requests/sec?
• Scan Time for searching, data mining, disk utilities & RAID rebuild?

Disk drives have 3 different dimensions: WxDxH and 3 different 'faces', WxD, WxH, DxH
For 2.5" drives, approx 70mm x 101mm x 9.5mm
For 3.5" drives, 101.6mm (4 inch) x 146mm (6inch) x 19-26.1mm (nominally "1 inch")

Drives can be placed with any of the 3 faces down and rotated about a vertical axis, giving potentially 6 orientations.
In practice, the thinest cross-section has to face forward, into the airflow, to allow effective cooling.
This gives just 4 orientations: 2 'flat' and 2 'vertical'.

19 inch racks are "mostly standard":

• 19 inches across the faceplate, posts are each 5/8 inch, fasteners & holes are well defined.
• But need extra space either side for cabling and airflow, increasing external rack dimension.
• 17.75 inches internal clearance (450mm). With sliders: 17.25 inches internal. (435mm)
• 1RU (Rack Unit) = 1 .75 inches high
• convention is 42RU high = 73.5 inches of usable space
• Allow for plinth, first usable RU is off the floor
• Allow for head piece, plate + structural rails,fans and cable organisers on top
• Depth varies on use:
• 600mm (24 inch) common in Telecoms
• 966mm (38 inch) common in IT.
• Need extra space front and rear for doors, cabling, power strips, ...
• External dimensions: 30in x 48in x 87in (WxDxH)
• Notionally, a single rack uses ten square feet (1 square meter) of floor space.
• side clearance of zero: racks bolt together to stabilise the structure.
• Front and rear clearance, often 40" and 30" are needed to open doors and load/unload parts.
• Aisles are needed between rows to allow work and access.
• In many facilities, need to open two doors at once, 50" minimum.
• "Hot Aisle": exhaust adjacent rows into the one sealed area with extractor fan.
• Floor space in server rooms
• Only around 33%-50% of the available floor space can be used for racks.
• Racks are best organised in rows parallel to long dimension of room
• long rooms need breaks in rows, creating cross-aisles
• Additional clearance is needed around walls of rooms
• Entrance doors need to be double and handle shipping pallets
• Extra spare space is needed around doors for staging equipment in, and storing packing waste before removal
• Dedicated space is needed for "Air handling units" (at least two), power distribution boards and fire control systems. These need clearance for servicing and removal/replacement.
• In room UPS units need space and cooling (No-break power supplies)
• lead-acid battery banks of any capacity need to be housing in separate, spark-proof rooms with additional fire control and sprinklers.

Stacking 3.5 inch drives, no allowance for cooling, wiring, power or access:

3.5" drives, at 4 inches wide can be stacked flat, 4 abreast in a rack.
6 drives will fit end-to-end in a 36"-38" cabinet, for 24 drives in a layer.
Alternatively, 17 drives can be stood on their sides across a rack, 4" tall layers.
With 102 drives/layer and 1836 drives/rack.

For nominal 1" thick drives, 72 layers can be stacked, giving 1,752 drives per rack.
With 15TB 3.5" drives, 26PB/rack.
With 4TB 3.5" drives, 7PB/rack.
7200 RPM 3.5" drives consume 8W-10W, or 14kW-16kW per rack.
7200 RPM, 120Hz, drives are capable each of 240 IO/sec, for 400k IO/sec aggregate.
3.5" drives weight ~600grams each, for a load of about 1 ton (or 1,000kg/m²)
15TB 3.5" drives will stream at around 2Gbps, for 3.5Tbps aggregate internal bandwidth.

Stacking 2.5" drives, at 5400 RPM (90Hz)

5mm	17,800 drives	3,204k IO/sec	@ 0.5TB 9PB/rack	@ 1.5TB 27PB/rack
7mm	12,714 drives	2,288k IO/sec
9.5mm	9,368 drives	1,686k IO/sec	@ 1TB 9.5PB/rack	@ 3TB 30PB/rack
12.5mm	7,120 drives	1,281k IO/sec
15mm	5,933 drives	1,069k IO/sec	@ 2TB 12PB/rack	@ 6TB 36PB/rack

Power consumption at 1.2W for 9.5mm drives of 8kW, around half the power needed for 3.5" drives.

Aggregate internal bandwidth is higher, even though the per-drive streaming rate is up to 25% lower, 1.5Gbps.
For 9.5mm drives, 14Tbps aggregate internal bandwidth (3TB drives).

5mm drives weigh around 95grams each and 15mm drives 200grams, the same weight ± 15% as 3.5" drives.