From Steeple

Contents 1 Proposed Hardware Cluster 1.1 Costing for New Components 1.2 Costs in relation to capacity 1.3 Physical Installation

[edit] 1 Proposed Hardware Cluster

To meet the minimum requirements for functionality and ease of future growth, with an emphasis on throughput over high availability, we propose combining our existing resources with compatible new resources to build the hardware cluster outlined below:

In essence, this is a two node Podcaster Producer system, supported by an XSAN 2 datastore accessed via a dedicated single-path low-latency fibre-channel SAN.

[edit] 1.1 Costing for New Components

As of 22-April-2009, costs for the hardware and implementation option(s) are:

SINGLE PATH SETUP
Purpose		Item	Quantity	Cost per item	Subtotal
RAID Storage Device
First choice		Competitor SAN	1	£10,105	£10,105
Second choice		Apple Certified SAN	0	£11,500	£0
XSAN
Metadata Controllers		X Serve	2	£2,465	£4,930
		Applecare Support Plan	0	£502	£0
		Applecare Service Parts Kit	1	£502	£502
XSAN Clients		XSAN 2	4	£342	£1,368
		XSAN Support Care 1year	1	£418	£418
Metadata LAN		Ethernet Switch	1	£70	£70
		Ethernet cables	13	£3	£39
Consultancy		2 Days	1	£1,000	£1,000
Fibre Channel Fabric
Switches		Qlogic SANBox Express 1400 10 Ports	1	£1,897	£1,897
		Fibre Cable	0	£52	£0
For existing xserves		4Gb FC PCI-Express 16 Addon	2	£331	£662
Podcasting Cluster
New Head Node		X Serve	1	£2,465	£2,465
		Applecare Support Plan	0	£502	£502
Totals					£23,456

DUAL PATH SETUP
Purpose		Item	Quantity	Cost per item	Subtotal
RAID Storage Device
First choice		Competitor SAN	1	£10,105	£10,105
Second choice		Apple Certified SAN	0	£11,500	£0
XSAN
Metadata Controllers		X Serve	2	£2,465	£4,930
		Applecare Support Plan	0	£502	£0
		Applecare Service Parts Kit	1	£502	£502
XSAN Clients		XSAN 2	4	£342	£1,368
		XSAN Support Care 1year	1	£418	£418
Metadata LAN		Ethernet Switch	1	£70	£70
		Ethernet cables	13	£3	£39
Consultancy		2 Days	1	£1,000	£1,000
Fibre Channel Fabric
Switches		Qlogic SANBox Express 1400 10 Ports	2	£1,897	£3,794
		Fibre Cable	4	£52	£207
For existing xserves		4Gb FC PCI-Express 16 Addon	2	£331	£662
Podcasting Cluster
New Head Node		X Serve	1	£2,465	£2,465
		Applecare Support Plan	0	£502	£502
Totals					£25,560

The largest single cost is the RAID Box that underpins the high-availability storage needed to power this solution. Our primary choice is kit from a manufacturer with whom we’ve been developing a good relationship and whose product is technically superior to the alternative current available.

XSAN requires two machines to be dedicated Metadata Controllers (MDCs) as these are critical nodes in the SAN infrastructure. Both machines need only to be basic specification machines, with the addition of Fibre Channel Cards and slightly increased memory. The SAN uses a private low-latency Ethernet connection for metadata traffic; this is provided by the cheap consumer level Ethernet switch detailed.

The key difference between the two shopping lists is the Single Point Of Failure (SPOF) that critically affects (see Risk Analysis section) all of the cluster’s operations – i.e. the data transfer layer: The fibre channel network. The single path setup provides only one network path and failures in either the cables or the switch itself will cripple the entire cluster. The Dual path list rectifies this problem by providing two independent fibre channel paths, both improving fault tolerance, and increasing capacity within the SAN. Whilst it is possible to implement the single path solution initially and upgrade to the dual path at a later date with minimal downtime, this is not the preferred option.

To ensure effective use of time and minimise risks caused by ignorance, it is highly recommended that a qualified and experienced engineer be brought in to assist with setup and some training. An experienced consultant has been identified as suitable and has worked with OUCS in the past. His costs are presently estimated and can’t be confirmed until equipment delivery dates and his availability have been aligned. It may also be possible to combine the installation consultation time with a bespoke training course delivered by the consultant for interested parties within Oxford, thus reducing the costs of doing these activities separately.

One option is to consider utilising the new XServe as the processing engine node rather than as the head node. Due to the recent refresh of the Apple product line and the performance improvements introduced, the new base model is likely to be almost equivalent in processing capability to our existing upgraded XServe. By adding approximately £500 to the ticket price we can upgrade to a dual core model and thus maintain the 8 core architecture for performance gains in parallelism, whilst benefiting from the faster processors and architecture (i.e. significant improvements in transcoding speeds and publishing throughput).

Applecare has been left off of the pricing as a sacrificial offering to reduce the initial cost of purchasing. Whilst this is against OUCS policy for production systems and not our recommended option, we are optimistic that the risks associated with the lack of cover (and the onsite availability of some spare parts) will not cost us excessively in time and resources later, should something physically fail (see Risk Analysis).

[edit] 1.2 Costs in relation to capacity

Unit	Cost (10)	Usable Capacity	£/Tb Usable Capacity
Preferred SAN 8Tb	£8699	3.4Tb	£2559
Preferred SAN 16Tb	£10869	10Tb	£1087
Preferred SAN 32Tb	£20468	20.4Tb	£1003
Alternate SAN 8Tb	£9499	3.4Tb	£2784
Alternate SAN 16Tb	£11499	10Tb	£1150
Alternate SAN 32Tb	£20998	20.4Tb	£1029

(10) The Preferred SAN prices quoted here are RRP, and do not include any negotiated discount. The Promise prices include maximum Educational discount from Apple.

Usable capacity is determined by the following formula: Total number of disks in stack (t). 2+1 Disks assigned to Metadata (t-3). Take the largest number of remaining disks (v). Divide v by the number of LUNS to be configured (e.g. 2), to give n. LUNS are likely to be RAID 5 (though for more than 5 disks in a LUN, RAID 6 may be more sensible), so capacity (c) is disk-size*(n-1). Any spare disk is used as a hot spare for main volumes. Combine c*l and then multiple by 0.85 (XSAN2 has performance issues when over 85% full), to give the total usable capacity under XSAN2.

Each of the above systems uses 1Tb drives. 8 disk solutions assume 1 LUN of 5 disks. 16 disk solutions assume 2 LUNS of 6 disks. 32 disk solutions assume 4 LUNS of 7 disks.

[edit] 1.3 Physical Installation

Solution requires 9U of standard rack space. This has been located in the OUCS Server Room within the cabinet currently dedicated to Apple hardware, and still leaves sufficient space for future expansion. Some further minor investigation is needed to ensure there are enough PDUs within the rack. All XServes and SAN equipment come supplied with suitable rack mounting kit.